Block chain storage space cleaning method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a block chain storage space cleaning method and device, equipment, a medium and a product, and relates to the field of financial science and technology or the field of block chains. The method comprises the following steps: after obtaining total data stored by a first node in a block chain, copying the total data to an archiving node; then, determining popularity data in the total data, and determining the size of a to-be-cleaned space of the first node; then, according to the size of the to-be-cleaned space, screening out other data except the hot data from the total data to determine to-be-cleaned data, and finally, deleting the to-be-cleaned data from a plurality of nodes of the block chain. According to the method, the safe backup of the data is ensured through the archiving node, and the technical problems of high storage cost and high storage capacity demand faced by a distributed storage mode in a block chain technology are effectively solved by optimizing the storage space of the node.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial technology or the field of blockchain, in particular to a blockchain storage space cleaning method, device, equipment, medium and product. BACKGROUND

[0002] With the rapid growth of data volume and the increasing frequency of digital transactions, blockchain technology has become an important tool for storing and managing large-scale data due to its unique security and transparency.

[0003] The existing blockchain adopts a distributed storage mode, which requires multiple nodes in the blockchain network to jointly save a complete data copy. In these nodes, each node plays a dual role of data storage and data verification, and these nodes can ensure that the data in the entire blockchain network is reliable and tamper-proof through consensus algorithms.

[0004] However, although the distributed storage mode significantly improves the reliability of data, since each node needs to save a complete data copy, it requires each node to be equipped with more hardware devices and larger bandwidth, and requires the node to have larger storage capacity. SUMMARY

[0005] The present application provides a blockchain storage space cleaning method, device, equipment, medium and product to solve the technical problems of high storage cost and high storage capacity demand faced by the distributed storage mode in blockchain technology.

[0006] In a first aspect, the present application provides a blockchain storage space cleaning method, comprising:

[0007] Obtaining full data from a first node of a blockchain, and copying the full data to an archival node, the first node being any one of a plurality of nodes of the blockchain;

[0008] Determining hot data in the full data, and determining a to-be-cleaned space size of the first node;

[0009] According to the to-be-cleaned space size, determining to-be-cleaned data from other data in the full data except the hot data;

[0010] Deleting the to-be-cleaned data from the plurality of nodes of the blockchain.

[0011] In a second aspect, the present application provides a blockchain storage space cleaning device, comprising:

[0012] An obtaining module is configured to obtain full data from a first node of a blockchain, the first node being any one of a plurality of nodes of the blockchain;

[0013] copying a full amount of data stored in a first node of a blockchain to an archiving node;

[0014] determining hot data in the full amount of data, and determining a size of a space to be cleaned in the first node;

[0015] The determining module is further configured to determine, according to the size of the space to be cleaned, data to be cleaned from other data in the full amount of data except the hot data.

[0016] deleting the data to be cleaned from a plurality of nodes of the blockchain.

[0017] In a third aspect, an embodiment of the present application provides a blockchain storage space cleaning device, comprising a memory and a processor.

[0018] The memory stores computer execution instructions.

[0019] The processor executes the computer execution instructions stored in the memory, so that the processor executes the first aspect and / or various possible implementation manners of the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0022] The blockchain storage space cleaning method, device, equipment, medium and product provided by the present application, after obtaining the full amount of data stored in the first node of the blockchain, first copies the full amount of data to the archiving node; then, the hot data in the full amount of data is determined, and the size of the space to be cleaned in the first node is determined; then, according to the size of the space to be cleaned, the data to be cleaned is determined from other data in the full amount of data except the hot data, and finally, the data to be cleaned is deleted from the plurality of nodes of the blockchain. The method not only ensures the safe backup of data through the archiving node, but also optimizes the storage space of the node, effectively solving the technical problems of high storage cost and high storage capacity demand faced by the distributed storage mode in the blockchain technology. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0024] Figure 1 Flowchart of the blockchain storage space cleaning method provided for the present application Figure 1 ;

[0025] Figure 2 Flowchart of the blockchain storage space cleaning method provided for the present application Figure 2 ;

[0026] Figure 3 Flowchart of the blockchain storage space cleaning method provided for the present application Figure 3 ;

[0027] Figure 4 Structure diagram of the blockchain storage space cleaning device provided for the present application

[0028] Figure 5 Structure diagram of the blockchain storage space cleaning device provided for the present application

[0029] The specific embodiments of the present application have been shown in the above-mentioned drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0030] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings in which like reference numerals refer to like elements, unless otherwise indicated. The following exemplary embodiments described in the following description are not meant to be limiting of the present application. Instead, they are meant to provide examples of apparatus and methods in accordance with aspects of the present application as detailed in the appended claims.

[0031] It should be noted that the blockchain storage space cleaning method, device, equipment, medium and product provided by the present application can be used in the field of financial technology or the field of blockchain, and can also be used in any field other than the field of financial technology or the field of blockchain. The application of the blockchain storage space cleaning method, device, equipment, medium and product provided by the present application is not limited.

[0032] Blockchain technology, as a decentralized distributed ledger technology, its core is to combine a series of data blocks in a chain way according to time sequence into a specific data structure, and use cryptography to ensure the security of data transmission and access. The information stored in these data blocks usually includes transaction records, timestamps, on-chain addresses and other key information, ensuring the data's non-tamperability and traceability. With the continuous expansion of blockchain applications, the amount of data stored is also growing rapidly, covering important information in finance, supply chain management, Internet of Things and other fields.

[0033] In the current blockchain technology system, data storage mainly relies on distributed storage. This method requires multiple nodes to jointly save complete data copies to ensure data reliability and redundancy. Each node plays the role of data storage and verification, and ensures data consistency and security through consensus mechanism. This distributed storage method not only improves the anti-attack ability of the blockchain system, but also enables efficient synchronization and sharing of data among different nodes, providing a solid foundation for the widespread application of blockchain technology.

[0034] However, although the distributed storage method significantly improves the reliability of data, since each node needs to save a complete data copy, it requires each node to be equipped with more hardware devices and larger bandwidth, and each node needs to have larger storage capacity.

[0035] To solve the above problems, the blockchain storage space cleaning method provided by the present application is provided. After obtaining the full amount of data stored by the first node in the blockchain, the full amount of data is first copied to the archive node; then, the hot data in the full amount of data is determined, and the size of the first node's cleaning space is determined; then, according to the size of the cleaning space, the cleaning data is determined from the other data in the full amount of data except the hot data; finally, the cleaning data is deleted from the multiple nodes of the blockchain. This method not only ensures the safe backup of data through the archive node, but also optimizes the storage space of the node, effectively solving the technical problems of high storage cost and high storage capacity demand faced by the distributed storage method in the blockchain technology.

[0036] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0037] Figure 1 The flowchart of the blockchain storage space cleaning method provided by the present application Figure 1 As shown in Figure 1As shown, the method comprises:

[0038] S101, obtaining full amount data from a first node of the blockchain, and copying the full amount data to an archival node, the first node being any node of the plurality of nodes of the blockchain.

[0039] The full amount data refers to the sum of all data stored on the blockchain, which represents the complete state of the blockchain.

[0040] The archival node is an off-chain node of the blockchain, responsible for backing up the full amount data of the blockchain.

[0041] The purpose of this step is to safely transfer and backup the on-chain data of the blockchain to the off-chain archival node.

[0042] It can be understood that in the blockchain, all nodes store the same data, which means that obtaining the full amount data from any on-chain node in the blockchain can obtain the sum of all data currently stored in the blockchain. Therefore, selecting any node in the blockchain as the first node and obtaining the full amount data from it not only improves the efficiency of data acquisition, but also ensures the integrity and consistency of the data, because all nodes store the same data.

[0043] The archival node is an off-chain node in the blockchain system, which is specifically used to backup the data on the chain. Therefore, by safely backing up the full amount data on the chain to the off-chain archival node, it can ensure that even if the on-chain data is cleaned up or deleted for some reason, the complete data copy is still retained in the archival node, providing a reliable basis for data recovery and verification.

[0044] For example, assume that the blockchain is a trade blockchain, which is composed of trade nodes numbered 1 to 10, and these 10 trade nodes all store the same 10 trade data. In this case, the trade node numbered 1 can be selected as the first node, and the full amount data can be determined as the 10 trade data commonly stored by the 10 trade nodes.

[0045] S102, determining the hot data in the full amount data, and determining the size of the to-be-cleaned space of the first node.

[0046] The hot data refers to the data in the full amount data of the blockchain that is frequently accessed or has high importance.

[0047] The size of the to-be-cleaned space is used to indicate how much storage space the first node currently needs to release.

[0048] By determining the hotness data in the full amount of data, it indicates that the frequently accessed or high importance data in the full amount of data can be screened out. And by determining the size of the space to be cleaned in the first node, it indicates that how much storage space in the first node needs to be deleted at present can be determined.

[0049] For example, assuming that the size of the space to be cleaned is 5MB, it can be determined that 5MB of storage space in the first node needs to be deleted at present.

[0050] S103, determining the data to be cleaned from the other data in the full amount of data except the hotness data according to the size of the space to be cleaned.

[0051] Among them, the data to be cleaned refers to the data that needs to be deleted in the full amount of data.

[0052] The purpose of this step is to ensure the release of storage space while not mistakenly deleting or missing the hotness data which has key value to the blockchain.

[0053] It can be understood that the size of the space to be cleaned indicates how much storage space in the first node needs to be released at present, and the hotness data indicates which data in the full amount of data is frequently accessed by users or has high importance.

[0054] Therefore, by taking the size of the space to be cleaned as a determining factor and determining the other data that needs to be deleted from the full amount of data except the hotness data, not only can the hotness data in the full amount of data be properly preserved, avoiding mistakenly deleting or missing information which has key value to the blockchain, but also can effectively delete other redundant data in the full amount of data except the hotness data, thereby releasing valuable storage space and ensuring that the blockchain system of the first node can continue to run stably and efficiently.

[0055] S104, deleting the data to be cleaned from multiple nodes of the blockchain.

[0056] Among them, the purpose of this step is to delete the data to be cleaned from the storage of multiple nodes of the blockchain after determining the data to be cleaned.

[0057] It can be understood that each node in the blockchain stores complete or partial blockchain data. However, as the blockchain continues to expand, the amount of data on the node also presents a continuous growth trend. This not only poses a challenge to the running efficiency of the blockchain, but also may pose a potential threat to the security of the data.

[0058] Therefore, by requiring each node in the blockchain to delete the previously determined data to be cleaned, not only can the performance decline of the node due to excessive data volume be effectively avoided, but also the data consistency and security of the entire blockchain network can be ensured.

[0059] The blockchain storage space cleaning method provided by the embodiment of the present application first acquires full data from any node of the blockchain, and copies the data to an archive node to ensure data security backup. Then, hot data in the full data, i.e., data that is frequently accessed or has important value, is identified, and the size of the cleaning space of the first node is determined. Then, based on the demand of the cleaning space, non-hot data is filtered from the full data and marked as cleaning data. Finally, the cleaning data is safely deleted from multiple nodes of the blockchain to release the storage space. The method acquires full data from any node of the blockchain and copies the data to an archive node, ensuring the security backup of the data. At the same time, by identifying hot data and based on the demand of the cleaning space of the node, the storage space of the node is effectively released, thereby solving the technical problems of high storage cost and high storage capacity demand faced by the distributed storage mode in the blockchain technology.

[0060] Figure 2 Flowchart of the blockchain storage space cleaning method provided by the present application Figure 2 As shown in Figure 1 the embodiment is based on the Figure 3 The embodiment is based on the

[0061] S201, acquiring full data from a first node of a blockchain, and copying the full data to an archive node, the first node being any node of multiple nodes of the blockchain.

[0062] The explanation and description of step S201 are similar to those of step S101 described above, and will not be repeated here.

[0063] S202, sorting the full data in descending order of the usage amount of each data in the full data, and determining a preset number of data in the sorted result as hot data.

[0064] The usage amount of the data is used to represent the degree of access of the data by the user. The higher the usage amount of the data, the more frequently the user accesses the data. Conversely, the lower the usage amount of the data, the less frequently the user accesses the data.

[0065] The preset number of data may be, for example, the top 30% of the full data. The present application does not make special limitations on this.

[0066] The purpose of this step is to identify and filter the data with the highest usage amount, i.e., hot data, from the full data.

[0067] It can be understood that first, all data is sorted from high to low according to the usage of each data in the full data, which can clearly reflect which data is frequently accessed by users. Then, according to the preset number, the data in the front of the sorting result is determined as the hot data, so that the data frequently accessed by users can be determined as the hot data.

[0068] For example, assuming that the preset number is the top 30% of the full data. It is known that there are 50 pieces of full data, and based on the above information, the top 15 pieces of full data can be determined as the hot data.

[0069] S203, determine the feature vector of each data in the full data.

[0070] The purpose of this step is to convert each data in the full data set into a feature vector.

[0071] It can be understood that the feature vector contains the key information of the data, which can reflect the essential properties of the data. Therefore, by converting the original data into a feature vector, not only can the dimension and complexity of the data be effectively reduced, but also the processing efficiency of the data can be improved, so that the subsequent data analysis can be more convenient and efficient.

[0072] S204, based on the feature vector of each data, clustering the full data, and determining the aggregation degree of the obtained class cluster.

[0073] The aggregation degree of the class cluster represents the access degree of the user to the data in the class cluster. The higher the aggregation degree, the more frequently the user accesses the data in the class cluster. On the contrary, the lower the aggregation degree, the less frequently the user accesses the data in the class cluster.

[0074] The purpose of this step is to cluster the full data in order to divide the data with similar features in the full data into the same class cluster.

[0075] In this step, for example, the full data can be clustered based on the feature vector of each data to obtain a plurality of class clusters, and then the aggregation degree of each class cluster is determined according to the plurality of class clusters.

[0076] It can be understood that since the feature vector of each data respectively contains the key information of the data, by taking the feature vector of each data as an analysis factor and using a clustering algorithm, it means that the data with similar features can be divided into the same class cluster. After the clustering process, the aggregation degree of the obtained class cluster needs to be evaluated to determine the access degree of the user to the data in the class cluster.

[0077] S205, for the first type of cluster with the highest degree of aggregation, determine the cluster center of the first type of cluster, and determine the data in the first type of cluster that is less than or equal to the preset distance from the cluster center as the hotness data.

[0078] The first type of cluster with the highest degree of aggregation is used to represent that the user frequently accesses the data in the cluster compared to the data in other clusters.

[0079] The cluster center of the first type of cluster refers to the data in the center position among all data in the cluster.

[0080] The preset distance may be, for example, 20% of the average distance within the first type of cluster. The present application does not make special restrictions on this.

[0081] The purpose of determining whether the distance between any data in the first type of cluster and the cluster center is less than or equal to the preset distance is to determine whether there is user frequently accessed data in the first type of cluster.

[0082] If the distance between any data in the first type of cluster and the cluster center is less than or equal to the preset distance, it indicates that there is user frequently accessed data in the first type of cluster. Therefore, any data needs to be determined as hotness data.

[0083] In this step, for example, the cluster center of the first type of cluster with the highest degree of aggregation can be determined first; then the distance between any data in the first type of cluster and the cluster center is determined; then it is determined whether the distance between any data in the first type of cluster and the cluster center is less than or equal to the preset distance; finally, in the case where the distance between any data in the first type of cluster and the cluster center is less than or equal to the preset distance, any data is determined as hotness data.

[0084] S206, determine the current available storage space of the first node.

[0085] The current available storage space refers to how much storage space is currently available in the first node.

[0086] For example, if it is known that the current available storage space of the first node is 30MB, it indicates that there is currently 30MB of storage space available in the first node.

[0087] S207, if the current available storage space is less than the preset available storage space threshold, determine the difference between the available storage space threshold and the current available storage space as the to-be-cleaned space size.

[0088] The preset available storage space threshold may be, for example, 10% of the total storage space of the first node. The present application does not make special restrictions on this.

[0089] For example, assuming that the preset available storage space threshold is 10% of the total storage space of the node, and the total storage space of the first node is known to be 500 MB. Then the available storage space threshold of the first node can be determined to be 50 MB.

[0090] The purpose of determining whether the current available storage space of the first node is less than the preset available storage space threshold is to determine whether the first node has sufficient available storage space.

[0091] If the current available storage space of the first node is less than the preset available storage space threshold, it indicates that the first node does not have sufficient available storage space, and therefore, the difference between the available storage space threshold and the current available storage space needs to be determined as the size of the to-be-cleaned space to ensure that the first node has sufficient available storage space.

[0092] It can be understood that when the current available storage space of the first node is lower than the preset available storage space threshold, it means that the storage space of the first node is insufficient. In order to ensure that the first node has sufficient storage space for use, the difference between the available storage space threshold and the current available storage space needs to be calculated and determined as the size of the to-be-cleaned space.

[0093] For example, assuming that the preset available storage space threshold is 1% of the storage space of the node. It is known that the total storage space of the first node is 1000 MB, and its current available storage space is 5 MB. Based on the above information, the available storage space threshold of the first node can be first determined to be 10 MB, then the current available storage space 5 MB can be compared with the available storage space threshold 10 MB to determine that the first node does not have sufficient available storage space, and finally, the difference (5 MB) between the available storage space threshold 10 MB and the current available storage space 5 MB is determined as the size of the to-be-cleaned space.

[0094] S208, determine other data in the full amount of data except for the heat data as candidate to-be-cleaned data.

[0095] Among them, by preferentially retaining the heat data and cleaning the non-heat data, the storage space can be effectively released and the overall performance and response speed of the system can be improved without seriously affecting the user experience.

[0096] It can be understood that in data processing and storage management, heat data usually represents information that users pay most attention to or frequently access, and these data are crucial for maintaining the normal operation of the system and user experience. In contrast, other data may have less impact on users in the short term, and therefore, when the storage space is limited or the storage efficiency needs to be optimized, other data become the objects that are preferentially considered for cleaning.

[0097] S209, determining, from the candidate data to be cleaned, the data to be cleaned with a size of a cleaning space based on the usage amount and the storage space occupation amount of each piece of data in the candidate data to be cleaned.

[0098] The storage space occupation amount is used to measure the size of the storage space occupied by the data in the node. If the storage space occupation amount of a certain data is larger, it indicates that the data occupies more storage space in the node. Conversely, if the storage space occupation amount of a certain data is smaller, it indicates that the data occupies less storage space in the node.

[0099] By the usage amount and the storage space occupation amount of each piece of data in the candidate data to be cleaned, it can be determined which data has the least impact on the system performance under the premise of meeting the business requirements, and which data can release the most storage space after being cleaned.

[0100] It can be understood that, first, the usage amount of data reflects the activity and actual demand of users for the data. If the usage amount of a piece of data is low, it may mean that it does not play an important role in the current or near-term business requirements, or the user's access frequency to it is low.

[0101] Second, the storage space occupation amount of data directly reflects the occupation of system resources by the data. The data with a large storage space occupation amount may have a negative impact on the system performance, even if its usage amount is not high, especially in the case of limited storage space.

[0102] Therefore, by comprehensively considering the usage amount and the storage space occupation amount of each piece of data, it can accurately identify from the candidate data to be cleaned those data that are neither important nor occupy a large amount of resources, and determine these data as the data to be cleaned.

[0103] For example, suppose there is a set of 100 pieces of candidate data to be cleaned, each of which has its usage amount and storage space occupation amount. Among them, the usage amount of data A is 1 time / month, and the storage space occupation amount is 100 MB; the usage amount of data B is 20 times / month, and the storage space occupation amount is 50 MB; the usage amount of data C is 1 time / month, and the storage space occupation amount is 200 MB. In the case of limited resources, it may be preferred to clean the data with low usage amount and large storage space occupation amount. Therefore, data C can be determined as the data to be cleaned.

[0104] S210, sending the index of the data to be cleaned to a plurality of nodes to perform a consensus algorithm on the data to be cleaned in the plurality of nodes to reach a consensus, and deleting the data to be cleaned from the plurality of nodes if the consensus is reached.

[0105] The purpose of sending the index of the data to be cleaned to the plurality of nodes in this step is to ensure that these nodes can accurately identify and locate the data that needs to be cleaned. The purpose of the plurality of nodes using a consensus algorithm to perform consistency consensus on the data to be cleaned is to ensure that all nodes can reach a consensus on the cleaning operation when processing the data to be cleaned.

[0106] It can be understood that due to network delays, failures and other reasons, the nodes in the distributed system may be in different states, resulting in differences in their cognition and processing of the data to be cleaned. Through the consensus algorithm, it can be ensured that all nodes can reach a consensus on the cleaning operation before deleting the data to be cleaned, thereby avoiding inconsistent or conflicting data.

[0107] This step deletes the data to be cleaned from the plurality of nodes in the case of consensus passing, in order to ensure the consistency and integrity of the data.

[0108] It can be understood that if a node deletes data without the consensus of other nodes, other nodes may still retain these data, resulting in inconsistent data. In addition, if the deletion operation is not supported by the consensus, there may be a risk of mistaken deletion or missed deletion, which will negatively affect the stability and reliability of the system. Therefore, performing the deletion operation in the case of consensus passing can ensure that all nodes can synchronously process the data to be cleaned, thereby maintaining the consistency and integrity of the data.

[0109] The blockchain storage space cleaning method provided in the embodiments of the present application first acquires full data from any node of the blockchain and copies the full data to an archive node to ensure data security. Then, the full data is sorted according to data usage, and a preset number of data with the highest usage is directly identified as hot data. Meanwhile, a feature vector of each piece of data in the full data is calculated, and the full data is analyzed by clustering based on the feature vectors to identify a cluster with the highest clustering degree, and data close to the clustering center in the cluster is also regarded as hot data. Next, the current available storage space of the first node is checked, and if the current available storage space is lower than a preset storage space threshold, the size of the space to be cleaned is calculated. On this basis, data other than the hot data in the full data is taken as candidate data to be cleaned, and the usage and storage space occupancy of the candidate data are comprehensively considered to select data matching the size of the space to be cleaned from the candidate data as the final data to be cleaned. Finally, the indexes of the data to be cleaned are sent to multiple nodes in the blockchain network, a consensus algorithm is used to ensure the consistency consensus of the data to be cleaned, and the data is safely deleted from the multiple nodes after the consensus is passed. The method acquires full data from any node of the blockchain and copies the full data to an archive node, ensuring comprehensive backup and safe storage of the data. Meanwhile, the method can intelligently select candidate data matching the size of the space to be cleaned as the final data to be cleaned based on the demand of the space to be cleaned, effectively releasing the storage space. In addition, the consensus algorithm is used to ensure the consistency consensus of the data to be cleaned, and the data is safely deleted from the multiple nodes after the consensus is passed, ensuring the data consistency and security of the blockchain network.

[0110] Figure 3 Flowchart of the blockchain storage space cleaning method provided in the present application Figure 2 . As shown in Figure 4 , the present embodiment is based on the Figure 4 embodiment and provides a detailed description of the implementation process of determining the data to be cleaned based on the usage and storage space occupancy of each piece of candidate data to be cleaned. The blockchain storage space cleaning method provided in the present embodiment includes the following steps.

[0111] S301, sort each piece of candidate data to be cleaned according to the usage from low to high, and sort a plurality of first candidate data to be cleaned occupying a first storage space size in sequence in front of the sorting according to the storage space occupancy of each piece of data from high to low, and sort the first candidate data to be cleaned occupying the space to be cleaned in sequence in front of the sorting, and determine the first candidate data to be cleaned as the data to be cleaned, wherein the first storage space size is greater than the size of the space to be cleaned.

[0112] The first storage space size is the sum of the storage space usage of each of the plurality of first candidate data to be cleaned. For example, the storage space usage of the first candidate data to be cleaned with index A is 3MB, and the storage space usage of the first candidate data to be cleaned with index B is 4MB. Then, it can be determined that the first storage space size is 7MB.

[0113] The purpose of ensuring that the first storage space size is greater than the cleaning space size is to determine that the first node has sufficient storage space available for use after completing data cleaning.

[0114] The purpose of this step is to determine data that is neither frequently used nor occupies a large amount of storage space from the candidate data to be cleaned.

[0115] It can be understood that by first sorting the data from low to high according to the usage, data that is used less frequently by users can be identified. Then, from these low-usage data, sorting from high to low according to the storage space usage can further filter out data that is not only low in usage but also occupies a large amount of storage space, so as to determine these data as data to be cleaned.

[0116] For example, assume that the cleaning space size is 25MB. It is known that there is a group of candidate data to be cleaned, including data A (occupying 5MB, usage 20 times), data B (occupying 15MB, usage 25 times), data C (occupying 35MB, usage 40 times), and data D (occupying 8MB, usage 90 times). Then, based on the above information, the data can be first sorted from low to high according to the usage, and the new order obtained is data A (20 times), data B (25 times), data C (40 times), and data D (90 times). Then, the data is sorted according to the storage space usage, and the new order obtained is data C (35MB), data B (15MB), data D (8MB), and data A (5MB). Then, according to the cleaning space size 25MB, data C is selected as the data to be cleaned, and the storage space usage 35MB occupied by data C is determined as the second storage space size.

[0117] S302, sort each data in the candidate data to be cleaned from high to low according to the storage space usage, and sort the plurality of second candidate data to be cleaned occupying the second storage space size in the front from low to high according to the usage of each data, and determine the second candidate data to be cleaned occupying the cleaning space size in the front as the data to be cleaned, wherein the second storage space size is greater than the cleaning space size.

[0118] The second storage space size is the sum of the storage space usage of each of the plurality of second candidate data. For example, the storage space usage of the second candidate data with index D is 6 MB, and the storage space usage of the first candidate data with index E is 8 MB, so the second storage space size can be determined as 14 MB.

[0119] The purpose of this step is to ensure that the cleaned data not only occupies a large storage space, but also is at a low level in the current or recent usage frequency, thereby maximizing the release of storage space without seriously affecting the normal use of the user or system.

[0120] It can be understood that, in the data optimization screening process, first, all candidate data to be cleaned is sorted from high to low according to the storage space usage, which can accurately identify which data occupies the most storage space. Then, in the sorted data, sort from low to high according to the usage, which can accurately identify which data is frequently accessed by the user; finally, based on the cleaning space size, select the data that meets the cleaning space size from the double sorted results, and the storage space usage of these data constitutes the second storage space, and these data are selected as the data to be cleaned.

[0121] For example, assuming that the cleaning space size is 15 MB. It is known that there is a group of candidate data to be cleaned, including data A (occupying 10 MB, usage 100 times), data B (occupying 8 MB, usage 50 times), data C (occupying 16 MB, usage 20 times), and data D (occupying 6 MB, usage 80 times). Then based on the above information, the data can be sorted first according to the storage space usage, and the new order obtained is data C (16 MB), data A (10 MB), data B (8 MB), and data D (6 MB); then, sort from low to high according to the usage, and the new order obtained is data C (20 times), data B (50 times), data D (80 times), and data A (100 times); then, according to the cleaning space size 15 MB, select data C as the data to be cleaned, and determine the storage space usage 16 MB occupied by data C as the second storage space size.

[0122] Optionally, the present application provides another possible implementation of the data to be cleaned based on the usage and storage space usage of each data in the candidate data to be cleaned, which includes:

[0123] First, sort the candidate data to be cleaned according to the data usage from high to low, and then sort each usage group from low to high according to the storage space usage, to obtain the sorted data result.

[0124] The double sorting indicates that the data occupying large storage space and having low usage can be filtered from the candidate data to be cleaned as the final data to be cleaned.

[0125] The purpose of this step is to double-sort the candidate data to be cleaned. First, the data is sorted in descending order according to the data usage, so that the data frequently accessed by the user is placed in the front. Then, the data is sorted in ascending order according to the storage space occupation, so that the data frequently accessed by the user and occupying small storage space is placed in the front.

[0126] For example, assuming that there are four candidate data to be cleaned, A (usage 100, storage space 10MB), B (usage 100, storage space 5MB), C (usage 80, storage space 8MB), and D (usage 60, storage space 6MB). Then, according to the sorting rule of the first step, the sorted data result is B (usage 100, storage space 5MB), A (usage 100, storage space 10MB), C (usage 80, storage space 8MB), and D (usage 60, storage space 6MB).

[0127] Second, according to the size of the cleaning space, a plurality of candidate data to be retained are selected from the sorted data result, which occupies the total storage space closest to but less than the size of the cleaning space.

[0128] The purpose of this step is to select one or more groups of data from the sorted data according to the size of the cleaning space, which occupies the total storage space closest to but less than the size of the cleaning space as the candidate data to be retained.

[0129] For example, assuming that the size of the cleaning space is 15MB. Then, the storage space occupation can be added from the sorted data in turn until a combination (B+C, occupying space 13MB) is found, which occupies the total storage space closest to but not more than 15MB. Therefore, B and C can be regarded as candidate data to be retained.

[0130] Third, a plurality of third candidate data to be cleaned are obtained by removing a plurality of candidate data to be retained from all candidate data to be cleaned, which occupies the third storage space size, wherein the third storage space size is equal to the total storage space occupied by all candidate data to be cleaned minus the total storage space occupied by the plurality of candidate data to be retained.

[0131] The candidate data to be retained selected in the second step is removed from all candidate data to be cleaned to obtain the remaining data to be cleaned, i.e., the third candidate data to be cleaned.

[0132] For example, assume that the size of the space to be cleaned is 15 MB. It is known that there are four candidate data to be cleaned, which are A (usage 100, storage space 10 MB), B (usage 100, storage space 5 MB), C (usage 80, storage space 8 MB) and D (usage 60, storage space 6 MB). And it is known that data B and data C are candidate data to be retained. Then based on the above information, data A and data D can be obtained as the third candidate data to be cleaned, and the total storage space size of data A and data D is 16 MB, which is greater than the size of the space to be cleaned 15 MB.

[0133] The training method of the blockchain storage space cleaning model provided by the embodiments of the present application first adopts two different sorting strategies for the candidate data to be cleaned to determine the final data to be cleaned. In the first strategy, the candidate data to be cleaned is first sorted from low to high according to the usage, and then a data subset occupying a first storage space greater than the size of the space to be cleaned is selected from the sorted data. The data in this subset is sorted from high to low according to the storage space occupation, and finally the data with the highest ranking and the total occupation equal to the size of the space to be cleaned is selected as the data to be cleaned. In the second strategy, the candidate data to be cleaned is first sorted from high to low according to the storage space occupation, and then a data subset occupying a second storage space greater than the size of the space to be cleaned is selected from the sorted data. The data in this subset is sorted from low to high according to the usage, and finally the data with the highest ranking and the total occupation equal to the size of the space to be cleaned is selected as the data to be cleaned. This method can more flexibly select the data to be cleaned, taking into account both the usage frequency of the data and the storage occupation of the data, thereby maximizing the release of storage space while ensuring the importance of the data.

[0134] Figure 5 The structure diagram of the blockchain storage space cleaning device provided by the present application is shown in Figure 5 The blockchain storage space cleaning device 400 provided by the embodiments of the present application includes:

[0135] The acquisition module 401 is configured to acquire full data from a first node of a blockchain, wherein the first node is any node of a plurality of nodes of the blockchain.

[0136] The replication module 402 is configured to replicate the full data to an archival node.

[0137] The determination module 403 is configured to determine hot data in the full data, and determine a size of a space to be cleaned of the first node.

[0138] The determination module 403 is further configured to determine, from other data in the full data except the hot data, data to be cleaned according to the size of the space to be cleaned.

[0139] The deleting module 404 is configured to delete the data to be cleaned from the plurality of nodes of the blockchain.

[0140] The blockchain storage space cleaning device provided in the embodiment can execute the method provided in the method embodiment, and has similar implementation principles and technical effects. Details are not described herein.

[0141] ​ A structural diagram of a blockchain storage space cleaning device provided in the present application is shown in FIG. 5. As shown in FIG. 5, the electronic device 500 provided in the embodiment includes at least one processor 501 and a memory 502. Optionally, the device 500 further includes a communication component 503. The processor 501, the memory 502, and the communication component 503 are connected through a bus 504. ​

[0142] In the implementation process, the at least one processor 501 executes the computer execution instructions stored in the memory 502, so that the at least one processor 501 executes the method described above.

[0143] The specific implementation process of the processor 501 can refer to the method embodiments described above, and has similar implementation principles and technical effects. Details are not described herein.

[0144] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0145] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), for example, at least one disk memory.

[0146] ​The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0147] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described above.

[0148] The present application also provides a computer readable storage medium, which stores computer execution instructions, and when a processor executes the computer execution instructions, the method described above is implemented.

[0149] The readable storage medium described above can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0150] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium, and can write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0151] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0153] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0154] If the function is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0155] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a computer readable storage medium. The program executes the steps of the above-mentioned method embodiments when executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0156] It should be noted that for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the application is not limited by the order of the described actions, because according to the application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0157] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0158] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0159] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0160] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBsM), Hybrid Memory Cube (HMC), etc.

[0161] If the integrated units / modules are implemented in the form of software program modules and sold or used as independent products, they can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0162] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0163] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The application is intended to cover any variations, uses or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains or can relate. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the application are indicated by the following claims.

[0164] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the appended claims.

Claims

1. A method for cleaning up blockchain storage space, characterized in that, include: The full data is obtained from the first node of the blockchain and copied to the archive node, where the first node is any one of the multiple nodes in the blockchain; Determine the heat data in the full data, and determine the size of the space to be cleaned in the first node; Based on the size of the space to be cleaned, determine the data to be cleaned from the other data in the full data, excluding the popularity data; The data to be cleaned is deleted from multiple nodes of the blockchain.

2. The method according to claim 1, characterized in that, Determining the popularity data in the full dataset includes: The full dataset is sorted from highest to lowest usage for each data point, and a predetermined number of data points at the top of the sort are identified as the popularity data.

3. The method according to claim 1, characterized in that, Determining the popularity data in the full dataset includes: Determine the feature vector of each data point in the full dataset; Based on the feature vector of each data point, the entire dataset is clustered, and the degree of clustering of the resulting clusters is determined. For the first type of cluster with the highest degree of aggregation, the cluster center of the first type of cluster is determined, and the data in the first type of cluster whose distance from the cluster center is less than or equal to a preset distance is determined as the heat data.

4. The method according to any one of claims 1-3, characterized in that, Determining the size of the space to be cleaned in the first node includes: Determine the current available storage space of the first node; If the current available storage space is less than a preset available storage space threshold, the difference between the available storage space threshold and the current available storage space is determined as the size of the space to be cleaned.

5. The method according to any one of claims 1-3, characterized in that, The step of determining the data to be cleaned from the full data (excluding the popularity data) based on the size of the space to be cleaned includes: All data in the full dataset other than the popularity data are identified as candidate data to be cleaned. Based on the usage and storage space occupied by each piece of data in the candidate data to be cleaned, the data to be cleaned with the required space size is determined from the candidate data to be cleaned.

6. The method according to claim 5, characterized in that, The step of determining the data to be cleaned, based on the usage and storage space occupancy of each data item in the candidate data to be cleaned, includes: Each piece of data in the candidate data to be cleaned is sorted from low to high according to its usage. Then, the first candidate data to be cleaned that occupies the first storage space size is sorted from high to low according to the storage space usage of each piece of data. The first candidate data to be cleaned that occupies the first storage space size is determined as the data to be cleaned, wherein the first storage space size is greater than the storage space size to be cleaned.

7. The method according to claim 5, characterized in that, The step of determining the data to be cleaned, based on the usage and storage space occupancy of each data item in the candidate data to be cleaned, includes: Each piece of data in the candidate data to be cleaned is sorted from high to low according to the storage space occupied. Then, the multiple second candidate data to be cleaned that occupy the second storage space size at the top of the sorting are sorted from low to high according to the usage of each piece of data. The second candidate data to be cleaned that occupy the space size to be cleaned at the top of the sorting are determined as the data to be cleaned, wherein the second storage space size is greater than the space size to be cleaned.

8. The method according to claim 5, characterized in that, The deletion of the data to be cleaned from multiple nodes of the blockchain includes: The index of the data to be cleaned is sent to the plurality of nodes, so that a consensus algorithm is used among the plurality of nodes to reach a consensus on the data to be cleaned, and the data to be cleaned is deleted from the plurality of nodes if the consensus is passed.

9. A blockchain storage space cleanup device, characterized in that, include: The acquisition module is used to acquire all data from the first node of the blockchain, where the first node is any one of the multiple nodes in the blockchain; The copy module is used to copy the full data to the archive node; The determination module is used to determine the heat data in the full data and to determine the size of the space to be cleaned in the first node; The determining module is further configured to determine the data to be cleaned from the data other than the popularity data in the full data according to the size of the space to be cleaned; The deletion module is used to delete the data to be cleaned from multiple nodes of the blockchain.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images