Block chain data archiving method and device, equipment and storage medium

Abstract

The invention provides a block chain data archiving method and device, equipment and a storage medium, and relates to the technical field of data archiving. The method comprises the following steps: acquiring first access frequencies of various node data; dividing the node data into hot data and warm data based on the first access frequency of each type of node data; storing the hot data in a local node, and migrating the warm data to a backup server; and according to the second access frequency of the temperature data in the backup server, dividing the temperature data with the second access frequency lower than a preset archiving threshold value into cold data, and migrating the cold data to a data archiving server. According to the method, three-level grading storage of the node data can be realized, the storage pressure and the storage cost of the node can be reduced possibly, the data query performance is improved, the data is divided according to the frequency, and the method can further meet the data calling requirement of a service scene.

Description

Technical Field

[0001] The present invention relates to the technical field of data archiving, and particularly to a blockchain data archiving method, device, equipment and storage medium. Background Art

[0002] Essentially, a blockchain is a multi-active distributed database across different locations, which stores all transaction information recognized by the network. Since each node in the blockchain needs to store data, as the volume of blockchain data increases day by day, there is a bottleneck in node storage, and data archiving is required. Data archiving is to transfer data that has not been used for a long time in the online database to the historical database or archive it in the form of files, reducing the scale of active data.

[0003] Currently, when archiving and storing blockchain node data, most of the existing data archiving technologies preset the data life cycle and classify and archive node data according to the data life cycle. Although this method can reduce the storage pressure of nodes, it cannot well adapt to the business scenarios of nodes and cannot achieve the optimal in terms of cost and performance. For example, for some data with a high usage frequency, after the end of its life cycle, it will be classified as other data and stored on other servers, but perhaps this data is actually still frequently used data, and such a data archiving method obviously cannot well meet the requirements of business scenarios. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a blockchain data archiving method, device, equipment and storage medium to solve the problem that the existing data archiving method for blockchain nodes has low applicability to business scenarios.

[0005] The specific technical solution of the present invention is as follows: In the first aspect of the embodiments of the present invention, a blockchain data archiving method is provided, including: Obtaining the first access frequency of various types of node data; Based on the first access frequency of various types of node data, dividing the node data into hot data and warm data; Storing the hot data locally on the node and migrating the warm data to the backup server; According to the second access frequency of the warm data in the backup server, dividing the warm data with the second access frequency lower than the preset archiving threshold into cold data and migrating the cold data to the data archiving server.

[0006] Further, the obtaining the first access frequency of various types of node data includes: During a first time period, analyze the content of the service requests received by the blockchain node, determine the service types of the service requests, and count the number of times the node data corresponding to each service type is accessed to obtain the first access frequency of the node data.

[0007] Further, based on the first access frequencies of various types of node data, divide the node data into hot data and warm data, including: During the first time period, compare the first access frequencies of various types of node data according to a preset backup threshold, use the node data with an access frequency greater than or equal to the preset backup threshold as hot data, and use the node data with an access frequency less than the preset backup threshold as warm data.

[0008] Further, storing the hot data locally on the node and migrating the warm data to a backup server includes: The blockchain node stores the hot data into the local first storage medium according to the service types of the node data in the hot data; the first storage medium is an SSD disk or memory; Use a preset data backup tool to migrate the warm data to the database of the backup server, and after the warm data is migrated, record the address of the migrated backup server, and at the same time delete the warm data locally on the node.

[0009] Further, based on the second access frequency of the warm data in the backup server, divide the warm data with a second access frequency lower than a preset archiving threshold into cold data, and migrate the cold data to a data archiving server, including: The backup server receives the warm data sent by the blockchain node and stores it in the second storage medium; the second storage medium is an HD hard disk; During a second time period, the backup server counts the second access frequencies of various types of warm data, compares the second access frequencies of various types of warm data according to a preset archiving threshold, and divides the warm data with a second access frequency lower than the preset archiving threshold into cold data; Use a preset data migration tool to migrate the cold data to the data archiving server for storage, and after the cold data is migrated, record the address of the migrated data archiving server, and at the same time delete the cold data locally on the backup server.

[0010] Further, the method according to an embodiment of the present invention further includes: if the second access frequency of the warm data counted by the backup server during the second time period is higher than the preset backup threshold, restore the corresponding warm data to the blockchain node.

[0011] In a second aspect, an embodiment of the present invention provides a blockchain data archiving device, including: A data acquisition module, configured to obtain the first access frequency of various types of node data; A data partitioning module, configured to partition node data into hot data and warm data based on the first access frequency of various types of node data; A data backup module, configured to store the hot data locally on the node and migrate the warm data to a backup server; A data archiving module, configured to partition, according to the second access frequency of the warm data in the backup server, the warm data with the second access frequency lower than a preset archiving threshold into cold data, and migrate the cold data to a data archiving server.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, including: One or more processors; A storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, enable the one or more processors to implement the blockchain data archiving method described in the first aspect.

[0013] In a fourth aspect, an embodiment of the present invention further provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, the blockchain data archiving method described in the first aspect is implemented.

[0014] The beneficial effects of the present invention are as follows: The method partitions node data into hot data and warm data based on the first access frequency of various types of node data in the blockchain node, stores the hot data locally on the node, and migrates the warm data to a backup server. At the same time, according to the second access frequency of the warm data in the backup server, cold data is further partitioned from the warm data, and the cold data is migrated to a data archiving server, realizing three-level hierarchical storage of node data, which may reduce the storage pressure and storage cost of the node, improve the data query performance, and at the same time, partitioning data according to the frequency can further meet the data call requirements of the business scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0016] Figure 1 is a flowchart of a blockchain data archiving method proposed by an embodiment of the present invention; Figure 2 is a schematic diagram of the module structure of a blockchain data archiving device proposed by an embodiment of the present invention; Figure 3 is a schematic diagram of the principle of data archiving proposed by an embodiment of the present invention. Specific Embodiments

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0018] Embodiment 1: Figure 1 It is a flowchart of a blockchain data archiving method proposed in an embodiment of the present invention. Referring to Figure 1 as shown, a blockchain data archiving method in an embodiment of the present invention can be applied to any node in a blockchain network, and mainly includes the following steps: S1: Obtain the first access frequency of various types of node data; S2: Based on the first access frequency of various types of node data, divide the node data into hot data and warm data; S3: Store the hot data locally on the node and migrate the warm data to a backup server; S4: According to the second access frequency of the warm data in the backup server, divide the warm data with the second access frequency lower than a preset archiving threshold into cold data, and migrate the cold data to a data archiving server.

[0019] Further, in one embodiment, obtaining the first access frequency of various types of node data in step S1 of the embodiment of the present invention specifically includes the following content: Within a first time period, analyze the content of the service requests received by the blockchain node, determine the service type of the service request, and count the number of times the node data corresponding to each service type is accessed to obtain the first access frequency of the node data.

[0020] In the embodiment of the present invention, the blockchain node may receive service requests of multiple service types, thereby generating different service data. To facilitate subsequent data query, the present invention first analyzes the service type of the service request to determine the service type of the service request data, which is convenient for centralized storage of the service data of each service type.

[0021] Further, in one embodiment, step S2 of the embodiment of the present invention divides the node data into hot data and warm data based on the first access frequency of various types of node data, and specifically includes the following process: Within a first time period, compare the first access frequency of various types of node data according to a preset backup threshold, and regard the node data with an access frequency greater than or equal to the preset backup threshold as hot data, and regard the node data with an access frequency less than the preset backup threshold as warm data.

[0022] Among them, the preset backup threshold can be set according to the storage capacity and node query performance of the blockchain node, and can be specifically set according to the actual situation. For example, node A can store 10G of data, but the best free memory for node query is 4G. At this time, the preset backup threshold of the node can be set to 6G. When the data stored in the node exceeds or reaches 6G, the data migration and backup operation is started.

[0023] Furthermore, in one embodiment, in step S3 of the embodiment of the present invention, the hot data is stored locally on the node, and the warm data is migrated to the backup server, including but not limited to the following sub-steps: S31: The blockchain node stores the hot data into the first storage medium locally according to the business type of the node data in the hot data. Among them, the first storage medium is an SSD disk or memory. The SSD disk refers to a solid-state drive. A solid-state drive (SSD) is made of a solid-state electronic storage chip array and consists of a control unit and a storage unit (FLASH chip, DRAM chip). The SSD disk has a fast read and write speed and can meet the high-frequency read and write of hot data.

[0024] S32: Use a preset data backup tool to migrate the warm data to the database of the backup server. After the warm data is migrated, record the address of the migrated backup server, and at the same time delete the warm data on the local node.

[0025] Among them, the preset data backup tool can use data migration tools such as ETL tools, heterogeneous data source offline synchronization tools DataX, and DataStage. Specifically, it can be selected according to the data type of the node or the actual business requirements. The embodiment of the present invention does not make a limitation here.

[0026] Furthermore, in one embodiment, in step S4 of the present invention, according to the second access frequency of the warm data in the backup server, the warm data with the second access frequency lower than the preset archiving threshold is classified as cold data, and the cold data is migrated to the data archiving server, including but not limited to the following sub-steps: S41: The backup server receives the warm data sent by the blockchain node and stores it in the second storage medium. Among them, the second storage medium is an HD hard disk. The HD disk, also known as a hard disk drive (HDD), has a slower read and write speed compared to the SSD hard disk and is suitable for reading and writing warm data with a low access frequency but that can be used in the short term.

[0027] S42: During the second time period, the backup server counts the second access frequencies of various warm data, compares the second access frequencies of various warm data according to a preset archiving threshold, and classifies the warm data with a second access frequency lower than the preset archiving threshold as cold data.

[0028] S43: Use a preset data migration tool to migrate the cold data to the data archiving server for storage. After the cold data is migrated, record the address of the migrated data archiving server, and at the same time delete the cold data on the local backup server.

[0029] Specifically, a query interface is provided on the backup server to facilitate users to retrieve temperature data. In addition, the blockchain node stores the address of the backup server. When a user requests warm data from the node, the warm data server can be accessed through the address of the backup server to obtain the warm data. After the backup server classifies the cold data, it can also synchronize the cold data information to the blockchain node so that the node can access the data archiving server to retrieve the cold data.

[0030] At the same time, the data archiving server stores the cold data in the form of optical discs, magnetic tapes, cloud storage, etc. When a user retrieves the cold data, the cold data is obtained through the query interface provided by the data archiving server.

[0031] Furthermore, the method of the embodiment of the present invention further includes: if the second access frequency of the warm data statistically obtained by the backup server during the second time period is higher than the preset backup threshold, the corresponding warm data is restored to the blockchain node.

[0032] Specifically, the embodiment of the present invention supports the function of restoring archived data. Through the data migration tool, all or part of the warm data and cold data can be restored to the online database of the blockchain node.

[0033] Embodiment 2: Refer to Figure 2 As shown, the embodiment of the present invention provides a blockchain data archiving device, including: A data acquisition module, configured to obtain the first access frequency of various node data; A data classification module, configured to classify the node data into hot data and warm data based on the first access frequency of various node data; A data backup module, configured to store the hot data locally on the node and migrate the warm data to the backup server; A data archiving module, configured to classify the warm data with a second access frequency lower than the preset archiving threshold as cold data according to the second access frequency of the warm data in the backup server, and migrate the cold data to the data archiving server.

[0034] Specifically, refer to Figure 3As shown in the schematic diagram of data archiving, after the blockchain node divides the hot data and warm data through the data partitioning module, the data backup module is used to migrate the warm data to the backup server for storage. Then, the backup server further divides the cold data from the warm data according to the access frequency of the warm data, and uses the data archiving module to migrate the cold data to the data archiving server, realizing the three-level storage of node data, which can effectively reduce the storage pressure of the node and reduce the storage cost.

[0035] In the blockchain data archiving device according to the embodiment of the present invention, based on the first access frequency of various node data in the blockchain node, the node data is divided into hot data and warm data. The hot data is stored locally on the node, and the warm data is migrated to the backup server. At the same time, according to the second access frequency of the warm data in the backup server, the cold data is further divided from the warm data, and the cold data is migrated to the data archiving server, realizing the three-level hierarchical storage of node data, which may reduce the storage pressure and storage cost of the node, improve the data query performance, and at the same time, dividing the data according to the frequency can further meet the data call requirements of the business scenario.

[0036] Furthermore, the embodiment of the present invention also provides an electronic device, including: One or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the blockchain data archiving method in Embodiment 1.

[0037] Furthermore, the embodiment of the present invention also provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the blockchain data archiving method in Embodiment 1.

[0038] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

[0039] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements that are not expressly listed, or also includes elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0040] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference can be made to the relevant part of the method embodiment for the relevant content.

[0041] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention are all included within the protection scope of the present invention.

Claims

1. A blockchain data archiving method, characterized in that, Including: Obtaining the first access frequency of various types of node data; Based on the first access frequency of various types of node data, dividing the node data into hot data and warm data; Storing the hot data locally on the node and migrating the warm data to a backup server; According to the second access frequency of the warm data in the backup server, dividing the warm data with the second access frequency lower than the preset archiving threshold into cold data, and migrating the cold data to a data archiving server.

2. The blockchain data archiving method according to claim 1, wherein The obtaining the first access frequency of various types of node data includes: Within a first time period, analyzing the content of the service requests received by the blockchain node, determining the service type of the service requests, and counting the number of times the node data corresponding to each service type is accessed, to obtain the first access frequency of the node data.

3. The blockchain data archiving method according to claim 1, characterized in that, The dividing the node data into hot data and warm data based on the first access frequency of various types of node data includes: Within a first time period, comparing the first access frequency of various types of node data according to a preset backup threshold, taking the node data with the access frequency greater than or equal to the preset backup threshold as hot data, and taking the node data with the access frequency less than the preset backup threshold as warm data.

4. The blockchain data archiving method according to claim 1, wherein The storing the hot data locally on the node and migrating the warm data to a backup server includes: The blockchain node stores the hot data into a local first storage medium according to the service type of the node data in the hot data; the first storage medium is an SSD disk or memory; Using a preset data backup tool to migrate the warm data to the database of the backup server, and after the warm data is migrated, recording the address of the migrated backup server, and at the same time deleting the warm data locally on the node.

5. The blockchain data archiving method according to claim 1, wherein The dividing the warm data with the second access frequency lower than the preset archiving threshold into cold data according to the second access frequency of the warm data in the backup server, and migrating the cold data to a data archiving server includes: The backup server receives the warm data sent by the blockchain node and stores it in a second storage medium; the second storage medium is an HD hard disk; Within a second time period, the backup server counts the second access frequency of various types of warm data, and compares the second access frequency of various types of warm data according to a preset archiving threshold, and divides the warm data with the second access frequency lower than the preset archiving threshold into cold data; Using a preset data migration tool to migrate the cold data to the data archiving server for storage, and after the cold data is migrated, recording the address of the migrated data archiving server, and at the same time deleting the cold data locally on the backup server.

6. The blockchain data archiving method according to claim 1, wherein It also includes: If the second access frequency of the warm data counted by the backup server within the second time period is higher than the preset backup threshold, then restoring the corresponding warm data to the blockchain node.

7. A blockchain data archiving device, characterized in that, Including: A data collection module, used to obtain the first access frequency of various types of node data; A data division module, used to divide the node data into hot data and warm data based on the first access frequency of various types of node data; A data backup module, used to store the hot data locally on the node and migrate the warm data to a backup server; A data archiving module, configured to divide warm data with a second access frequency lower than a preset archiving threshold into cold data according to the second access frequency of the warm data in the backup server, and migrate the cold data to a data archiving server.

8. An electronic device, characterized in that, It includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the blockchain data archiving method according to any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the blockchain data archiving method according to any one of claims 1 to 6.

Images