Blockchain-based kv data processing method and device based on clustered storage

Abstract

The application can be used in the technical field of the application of blockchain technology in finance, and provides a KV data processing method and device based on blockchain cluster storage, and a corresponding method comprises the following steps: storing main data in KV data and slave data corresponding to the main data in a position with a physical address less than a threshold value; generating a database with a skip list structure according to the number of the main data, the number of the slave data and the offset of the stored physical address between the main data and the slave data corresponding thereto; and processing the KV data according to the database with the skip list structure. The KV data processing method and device based on blockchain cluster storage provided by the application avoid increasing time consumption caused by multiple reading of associated data in the application layer, and simultaneously allow independent query of the associated data.

Description

Technical Field

[0001] This application belongs to the field of database technology, and in particular to the field of database and blockchain technology, specifically relating to a KV data processing method and apparatus based on blockchain clustered storage. Background Technology

[0002] In existing technologies, there are generally two methods for storing, querying, merging, and linking data in the underlying key-value (KV) database of a blockchain. The first method stores linked data as independent KV data in the underlying KV database, performing multiple queries to retrieve the individual KV values. Clearly, this method requires multiple queries on linked data, increasing query time. The second method merges linked data into a single value for storage, retrieving linked data from the value during queries. Since linked data can also be queried independently, it still needs to be stored separately. This method requires storing the same data multiple times, increasing storage overhead and exacerbating the write amplification problem inherent in KV databases. Summary of the Invention

[0003] This invention can be used in the technical field of database technology applied in finance, and can also be used in any field other than finance. The KV data processing method and device based on blockchain clustered storage provided by this invention avoids the increased time consumption caused by multiple reads of related data at the application layer, while allowing independent querying of related data.

[0004] To address the technical problems in the background section of this application, the present invention provides the following technical solutions:

[0005] In a first aspect, the present invention provides a KV data processing method based on blockchain clustered storage, comprising:

[0006] The master data in the KV data and the corresponding slave data are stored in a physical address location that is less than a threshold.

[0007] A database with a skip table structure is generated based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data.

[0008] The KV data is processed based on the database with the skip table structure.

[0009] In one embodiment of the present invention, generating a database with a skip table structure based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data includes:

[0010] Generate the underlying ordered linked list in the skip list structure based on all master data and all slave data;

[0011] The number of levels of the skip list structure is determined based on the head node of the underlying ordered linked list.

[0012] The master data and the slave data are inserted into the corresponding positions of the skip list structure according to a preset probability, wherein the skip list structure is divided into a master data area and a slave data area.

[0013] In one embodiment of the present invention, multiple master data in the master data area are stored in a preset order; the KV data is processed according to the skip list structure database, including updating, querying and merging the KV data according to the skip list structure database.

[0014] In one embodiment of the present invention, updating or querying the KV data according to the skip table structure database includes:

[0015] In the skip list structure, the main data area of ​​the target key-value data is located, and the main data of the target key-value data is updated or queried;

[0016] Based on the master data area and the offset, locate the slave data area of ​​the target KV data, and update or query the slave data of the target KV data.

[0017] In one embodiment of the present invention, merging the KV data according to the skip list structure database includes:

[0018] In the skip list structure, the master data and slave data of the KV data to be merged in the current layer's ordered linked list are moved to the corresponding ordered linked list in the lower layer according to the master data sequence number, and the master data sequence number of the corresponding ordered linked list in the lower layer is changed.

[0019] In one embodiment of the present invention, merging the KV data according to the skip list structure database includes:

[0020] When the KV data to be merged includes non-master-slave data, the master data and slave data are written to the first temporary file;

[0021] Write the non-master-slave data to the second temporary file;

[0022] The first temporary file and the second temporary file are merged and moved to the lower level of the ordered linked list where the KV data to be merged is located.

[0023] Secondly, the present invention provides a KV data processing device based on blockchain clustered storage, the device comprising:

[0024] The KV data storage module is used to store the master data and the corresponding slave data in the KV data at a physical address less than a threshold.

[0025] The database generation module is used to generate a database with a skip table structure based on the number of master data, the number of slave data, and the offset of the physical address of the master data and its corresponding slave data.

[0026] The KV data processing module is used to process the KV data according to the database with the skip list structure.

[0027] In one embodiment of the present invention, the database generation module includes:

[0028] The underlying table generation unit is used to generate the underlying ordered linked list in the skip table structure based on all the master data and all the slave data.

[0029] A layer number determination unit is used to determine the layer number of the skip list structure based on the head node of the bottom ordered linked list.

[0030] The data insertion unit is used to insert the master data and the slave data into the corresponding positions of the skip list structure according to a preset probability, wherein the skip list structure is divided into a master data area and a slave data area.

[0031] In one embodiment of the present invention, multiple master data in the master data area are stored in a preset order; the KV data processing module includes:

[0032] The KV data processing unit is used to update, query, and merge the KV data based on the skip table structure database.

[0033] In one embodiment of the present invention, the KV data processing unit includes:

[0034] The master data query unit is used to locate the master data area of ​​the target KV data in the skip list structure, and update or query the master data of the target KV data.

[0035] The data query unit is used to locate the slave data area of ​​the target KV data based on the master data area and the offset, and to update or query the slave data of the target KV data.

[0036] In one embodiment of the present invention, the KV data processing unit includes:

[0037] The data merging unit is used in the skip list structure to move the master data and slave data of the KV data to be merged in the current layer's ordered linked list to the corresponding ordered linked list in the lower layer according to the master data sequence number, and change the master data sequence number of the corresponding ordered linked list in the lower layer.

[0038] In one embodiment of the present invention, the KV data processing unit includes:

[0039] The first file generation unit is used to write the master data and slave data to the first temporary file when the KV data to be merged includes non-master-slave data;

[0040] The second file generation unit is used to write the non-master-slave data to the second temporary file;

[0041] The file merging unit is used to merge the first temporary file and the second temporary file and move them to the lower level of the ordered linked list where the KV data to be merged is located.

[0042] Thirdly, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a KV data processing method based on blockchain clustered storage.

[0043] Fourthly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a KV data processing method based on blockchain clustered storage.

[0044] Fifthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a KV data processing method based on blockchain clustered storage.

[0045] As described above, embodiments of the present invention provide a KV data processing method and apparatus based on blockchain clustered storage, comprising: firstly, storing the master data and the corresponding slave data in the KV data at a physical address less than a threshold; then, generating a database with a skip list structure based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data; and finally, processing the KV data based on the skip list structure database.

[0046] This invention provides a method and apparatus for processing key-value (KV) data based on blockchain clustered storage. It stores related data physically in close proximity, such as within the same data block or adjacent data blocks, and records the offset between master and slave data within the related data. When querying master data, slave data can be found in the nearest possible location based on the offset. Both master and slave data support independent queries, but slave data needs to record its first data block and its offset within that first data block for location during querying. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the KV data processing method based on blockchain clustered storage in an embodiment of the present invention. Figure 1 ;

[0049] Figure 2 This is a flowchart illustrating step 200 of the KV data processing method based on blockchain clustered storage in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the skip table structure in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the master-slave data area division in an embodiment of the present invention;

[0052] Figure 5 This is a flowchart illustrating step 300 of the KV data processing method based on blockchain clustered storage in an embodiment of the present invention.

[0053] Figure 6 This is a flowchart illustrating step 301 of the KV data processing method based on blockchain clustered storage in an embodiment of the present invention. Figure 1 ;

[0054] Figure 7 This is a flowchart illustrating step 301 of the KV data processing method based on blockchain clustered storage in an embodiment of the present invention. Figure 2 ;

[0055] Figure 8 This is a flowchart illustrating step 301 of the KV data processing method based on blockchain clustered storage in an embodiment of the present invention. Figure 3 ;

[0056] Figure 9This is a flowchart illustrating the KV data processing method based on blockchain clustered storage in a specific embodiment of the present invention.

[0057] Figure 10 This is a schematic diagram illustrating the master-slave data area division in a specific embodiment of the present invention;

[0058] Figure 11 This is a schematic diagram of data merging (before merging) in a specific embodiment of the present invention;

[0059] Figure 12 This is a schematic diagram of data merging (after merging) in a specific embodiment of the present invention;

[0060] Figure 13 This is a block diagram of a KV data processing device based on blockchain clustered storage in an embodiment of the present invention;

[0061] Figure 14 This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] It should be noted that the terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0065] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0066] The embodiments of the present invention provide a specific implementation of a KV data processing method based on blockchain clustered storage, see [link to relevant documentation]. Figure 1 The method specifically includes the following:

[0067] Step 100: Store the master data in the KV data and the corresponding slave data in a physical address location less than a threshold;

[0068] Step 200: Generate a database with a skip table structure based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data.

[0069] Step 300: Process the KV data according to the database with the skip list structure.

[0070] To address the issue of reading and writing related data in blockchain key-value (KV) databases, this invention proposes a blockchain KV database storage method based on clustered storage. This method stores related data in adjacent physical locations, avoiding the increased time consumption caused by multiple reads of related data at the application layer, while also allowing independent queries of related data.

[0071] In some embodiments, see Figure 2 In the KV data processing method based on blockchain clustered storage, step 200 includes:

[0072] Step 201: Generate the underlying ordered linked list in the skip list structure based on all the master data and all the slave data;

[0073] See Figure 3 In a skip list structure, each level is an ordered linked list containing a head and a tail, and can be in ascending or descending order. The bottom level (Level 1) contains all elements, and the remaining levels contain fewer elements than the level below it. The number of levels in which each element exists is determined by a given probability. If the probability is i, then the probability that an element exists in the i-th level is p(i) = k. i-1 If an element exists in a higher-level linked list, then that element must also exist in the next lower-level linked list.

[0074] In addition, such as Figure 4 As shown, the skip list divides the data into master-slave data areas and other data areas. Within the master-slave data areas, data is stored in an ordered manner. Between master and slave data areas, the master data area comes first, followed by the slave data area. The storage of other data areas is ordered with the master data area, but is independent of the slave data area.

[0075] Step 202: Determine the number of levels of the skip list structure based on the head node of the bottom ordered linked list;

[0076] Step 203: Insert the master data and the slave data into the corresponding positions of the skip list structure according to a preset probability, wherein the skip list structure is divided into a master data area and a slave data area.

[0077] For each element to be inserted, the level of that element is randomly generated based on probability. The level of each node is a random value, but the value does not exceed the maximum level of the current skip list. Starting from the head node, find the position to insert the element at each level. In each level, find the first node larger than the element to be inserted from left to right, and then move the pointer of that node down to the next level. In each level, insert the new element to the left of the current node. Then, move the pointer of the new element to the right of the current node.

[0078] Additionally, the skip list level can be dynamically adjusted. When inserting a node, if the new node's level is greater than the current skip list level, the skip list level needs to be dynamically increased. Similarly, when deleting a node, if the deleted node is the only node at the highest level, the skip list level needs to be dynamically decreased.

[0079] In some embodiments, see Figure 5 In the KV data processing method based on blockchain clustered storage, step 300 includes:

[0080] Step 301: Update, query, and merge the KV data according to the database with the skip table structure.

[0081] That is, the processing methods for KV data in step 300 include three types: update, query, and merge.

[0082] In some embodiments, see Figure 6 The method for updating or querying the key-value data based on the skip table structure database includes the following steps:

[0083] Step 3011: In the skip list structure, locate the main data area of ​​the target KV data, and update or query the main data of the target KV data;

[0084] Step 3012: Locate the slave data area of ​​the target KV data based on the master data area and the offset, and update or query the slave data of the target KV data.

[0085] In steps 3011 and 3012, when updating master-slave data, the master data area is located first, the master data area is updated, and then the pointer to the corresponding slave data area is obtained and then the slave data area is updated.

[0086] When querying master-slave data, first locate the master data area, find the master data, then obtain the pointer to the corresponding slave data area, and then find the slave data in the slave data area.

[0087] In some embodiments, see Figure 7 Merging the KV data according to the skip list structure database includes:

[0088] Step 3013: In the skip list structure, move the master data and slave data of the KV data to be merged in the current layer's ordered linked list to the corresponding ordered linked list in the lower layer according to the master data sequence number, and change the master data sequence number of the corresponding ordered linked list in the lower layer.

[0089] In some embodiments, see Figure 8 Merging the KV data according to the skip list structure database includes:

[0090] Step 3014: When the KV data to be merged includes non-master-slave data, write the master data and slave data to the first temporary file;

[0091] Step 3015: Write the non-master-slave data to the second temporary file;

[0092] Step 3016: Merge the first temporary file and the second temporary file, and move them to the lower level of the ordered linked list where the KV data to be merged is located.

[0093] In steps 3014 to 3016, during file merging, the adjacency relationship between master and slave data must be maintained. In the files to be merged, master and slave data areas must be adjacent; what needs to be processed are non-master and slave data. Files are merged according to the following method: if master and slave data exist in the files to be merged, the master and slave data are first written to temporary file 1; if non-master and slave data exists in the files to be merged, the non-master and slave data are written to temporary file 2.

[0094] Data written to temporary files is merged as follows: For internal keys with the same application key but different sequence numbers, the internal key with the largest sequence number is guaranteed to be written to the target temporary file; for internal keys with other sequence numbers, it is checked whether a snapshot exists, and if so, it is also written to the target temporary file; otherwise, it is considered that no read operation has been performed and is discarded. Internal keys marked for deletion are also discarded. Finally, temporary file 1 and temporary file 2 are merged to obtain the final merged file.

[0095] To address the issue of reading and writing related data in blockchain key-value (KV) databases, this invention proposes a blockchain KV database storage method based on clustered storage. This method stores related data in adjacent physical locations, avoiding the increased time consumption caused by multiple reads of related data at the application layer, while also allowing independent queries of related data.

[0096] See Figure 9 In one specific embodiment, the present invention also provides a specific implementation of a KV data processing method based on blockchain clustered storage, which specifically includes the following:

[0097] S1: Identify associated data in the target blockchain.

[0098] S2: Build a key-value database based on the associated data.

[0099] See Figure 10 The data block is divided into master-slave data areas and other data areas. Within the master-slave data areas, data is stored in an orderly manner. Between master and slave data areas, the master data area comes first, followed by the slave data area.

[0100] When querying master-slave data, first locate the master data area, find the master data, then obtain the offset of the corresponding slave data area, and then find the slave data in the slave data area.

[0101] S3: Convert the KV database into a skip list structure.

[0102] S4: File merging.

[0103] See Figure 11 as well as Figure 12 File merging combines the data files of level-n and level-n+1, ​​and the resulting file is written to level-n+1.

[0104] The above image shows the distribution of data files that need to be merged. The data files in the blue box in the image above need to be merged into the gray data files in the image below.

[0105] As can be seen from the above description, the KV data processing method based on blockchain clustered storage provides the following intended effects by expressing the design requirements clearly in the design language and specifications through smart contracts:

[0106] Based on the same inventive concept, this application also provides a KV data processing device based on blockchain clustered storage, which can be used to implement the method described in the above embodiments, as shown in the following embodiments. Since the principle of solving the problem by the KV data processing device based on blockchain clustered storage is similar to that of the KV data processing method based on blockchain clustered storage, the implementation of the KV data processing device based on blockchain clustered storage can refer to the implementation of the KV data processing method based on blockchain clustered storage, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0107] Embodiments of the present invention provide a specific implementation of a blockchain-based clustered storage KV data processing device capable of implementing a blockchain-based clustered storage KV data processing method. See [link to specific implementation details]. Figure 13 The KV data processing device based on blockchain clustered storage specifically includes the following:

[0108] KV data storage module 10 is used to store the master data in KV data and the slave data corresponding to the master data in a physical address location less than a threshold.

[0109] Database generation module 20 is used to generate a database with a skip table structure based on the number of master data, the number of slave data, and the offset of the physical address of the master data and its corresponding slave data.

[0110] KV data processing module 30 is used to process the KV data according to the database with the skip table structure.

[0111] In one embodiment of the present invention, the database generation module includes:

[0112] The underlying table generation unit is used to generate the underlying ordered linked list in the skip table structure based on all the master data and all the slave data.

[0113] A layer number determination unit is used to determine the layer number of the skip list structure based on the head node of the bottom ordered linked list.

[0114] The data insertion unit is used to insert the master data and the slave data into the corresponding positions of the skip list structure according to a preset probability, wherein the skip list structure is divided into a master data area and a slave data area.

[0115] In one embodiment of the present invention, multiple master data in the master data area are stored in a preset order; the KV data processing module includes:

[0116] The KV data processing unit is used to update, query, and merge the KV data based on the skip table structure database.

[0117] In one embodiment of the present invention, the KV data processing unit includes:

[0118] The master data query unit is used to locate the master data area of ​​the target KV data in the skip list structure, and update or query the master data of the target KV data.

[0119] The data query unit is used to locate the slave data area of ​​the target KV data based on the master data area and the offset, and to update or query the slave data of the target KV data.

[0120] In one embodiment of the present invention, the KV data processing unit includes:

[0121] The data merging unit is used in the skip list structure to move the master data and slave data of the KV data to be merged in the current layer's ordered linked list to the corresponding ordered linked list in the lower layer according to the master data sequence number, and change the master data sequence number of the corresponding ordered linked list in the lower layer.

[0122] In one embodiment of the present invention, the KV data processing unit includes:

[0123] The first file generation unit is used to write the master data and slave data to the first temporary file when the KV data to be merged includes non-master-slave data;

[0124] The second file generation unit is used to write the non-master-slave data to the second temporary file;

[0125] The file merging unit is used to merge the first temporary file and the second temporary file and move them to the lower level of the ordered linked list where the KV data to be merged is located.

[0126] As described above, embodiments of the present invention provide a KV data processing device based on blockchain clustered storage, comprising: firstly, storing the master data and the corresponding slave data in the KV data at a physical address less than a threshold; then, generating a database with a skip list structure based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data; and finally, processing the KV data based on the skip list structure database.

[0127] This invention provides a key-value (KV) data processing device based on blockchain clustered storage. It stores related data physically in close proximity, such as within the same data block or adjacent data blocks, and records the offset between master and slave data within the related data. When querying master data, slave data can be found in the nearest possible location based on the offset. Both master and slave data support independent queries, but slave data needs to record its first data block and its offset within that first data block for location during querying.

[0128] The embodiments of this application also provide a specific implementation of an electronic device capable of implementing all steps in the blockchain-based clustered storage KV data processing method described in the above embodiments. See [link to implementation details]. Figure 14 The electronic devices specifically include the following:

[0129] Processor 1201, memory 1202, communications interface 1203, and bus 1204;

[0130] The processor 1201, memory 1202, and communication interface 1203 communicate with each other via bus 1204; the communication interface 1203 is used to realize information transmission between server-side devices and client-side devices and other related devices.

[0131] The processor 1201 is used to call the computer program in the memory 1202. When the processor executes the computer program, it implements all the steps in the KV data processing method based on blockchain clustered storage in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0132] Step 100: Store the master data in the KV data and the corresponding slave data in a physical address location less than a threshold;

[0133] Step 200: Generate a database with a skip table structure based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data.

[0134] Step 300: Process the KV data according to the database with the skip list structure.

[0135] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the blockchain-based clustered storage KV data processing method in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the blockchain-based clustered storage KV data processing method in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0136] Step 100: Store the master data in the KV data and the corresponding slave data in a physical address location less than a threshold;

[0137] Step 200: Generate a database with a skip table structure based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data.

[0138] Step 300: Process the KV data according to the database with the skip list structure.

[0139] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0140] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0141] While this application provides method operation steps as shown in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the method can be executed sequentially as shown in the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0142] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this specification, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0143] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0144] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0145] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0146] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0147] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0148] The above description is merely an embodiment of the present specification and is not intended to limit the embodiments of the present specification. For those skilled in the art, various modifications and variations can be made to the embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present specification should be included within the scope of the claims of the embodiments of the present specification.

Claims

1. A method for processing key-value (KV) data based on blockchain clustered storage, characterized in that, include: The master data in the KV data and the corresponding slave data are stored at a physical address offset less than a threshold. A database with a skip table structure is generated based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data. The KV data is processed according to the database with the skip table structure; The step of generating a database with a skip table structure based on the number of master data, the number of slave data, and the offset of the physical address between the master data and its corresponding slave data includes: Generate the underlying ordered linked list in the skip list structure based on all master data and all slave data; The number of levels of the skip list structure is determined based on the head node of the underlying ordered linked list. The master data and the slave data are inserted into the corresponding positions of the skip list structure according to a preset probability, wherein the skip list structure is divided into a master data area and a slave data area.

2. The KV data processing method according to claim 1, characterized in that, The master data in the master data area is stored in a preset order; Processing the key-value data according to the skip list structure database includes updating, querying, and merging the key-value data according to the skip list structure database.

3. The KV data processing method according to claim 2, characterized in that, Updating or querying the KV data based on the skip table structure database includes: In the skip list structure, the main data area of ​​the target key-value data is located, and the main data of the target key-value data is updated or queried; Based on the master data area and the offset, locate the slave data area of ​​the target KV data, and update or query the slave data of the target KV data.

4. The KV data processing method according to claim 2, characterized in that, Merging the KV data according to the skip list structure database includes: In the skip list structure, the master data and slave data of the KV data to be merged in the current layer's ordered linked list are moved to the corresponding ordered linked list in the lower layer according to the master data sequence number, and the master data sequence number of the corresponding ordered linked list in the lower layer is changed.

5. The KV data processing method according to claim 2, characterized in that, The KV data is merged according to the skip list structure of the database, including: When the KV data to be merged includes non-master-slave data, the master data and slave data are written to the first temporary file; Write the non-master-slave data to the second temporary file; The first temporary file and the second temporary file are merged and moved to the lower level of the ordered linked list where the KV data to be merged is located.

6. A KV data processing device based on blockchain clustered storage, characterized in that, include: The KV data storage module is used to store the master data in the KV data and the corresponding slave data at a physical address offset less than a threshold. The database generation module is used to generate a database with a skip table structure based on the number of master data, the number of slave data, and the offset of the physical address of the master data and its corresponding slave data. The KV data processing module is used to process the KV data according to the database with the skip table structure. The database generation module includes: The underlying table generation unit is used to generate the underlying ordered linked list in the skip table structure based on all the master data and all the slave data. A layer number determination unit is used to determine the layer number of the skip list structure based on the head node of the bottom ordered linked list. The data insertion unit is used to insert the master data and the slave data into the corresponding positions of the skip list structure according to a preset probability, wherein the skip list structure is divided into a master data area and a slave data area.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the KV data processing method based on blockchain clustered storage as described in any one of claims 1 to 5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the KV data processing method based on blockchain clustered storage as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the KV data processing method based on blockchain clustered storage as described in any one of claims 1 to 5.

Images