Data storage optimization method based on block chain

Abstract

The invention discloses a block chain-based data storage optimization method. The method comprises the following steps of 1, establishing a storage strategy and a continuous mechanism for optimizing storage; 2, according to a storage strategy, calculating storage cost caused by monitoring real-time change of a storage amount statistical value and storage resource allocation service calculation; 3, continuously optimizing data storage in the whole data life cycle by utilizing the object label and the life cycle strategy; step 4, synchronizing transfer optimization of the data storage layer based on the access frequency of the data storage block; the method is applied to a data storage optimization system based on the block chain, the system comprises a storage strategy mechanism module, a storage cost calculation module and a data storage optimization module, in cloud storage, an intermediate data set is used for carrying out storage layer conversion decision making on data through the change of access frequency, and the cloud storage cost is saved. The method has the characteristics of improving storage performance and reducing storage cost.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a data storage optimization method based on blockchain. Background Art

[0002] After the completion of large-scale business calculations, existing enterprises will generate millions of data. These millions of data need to be stored in the database. However, due to the database timeout limit and the limit on the amount of data submitted at a single time, millions of data cannot be stored in the database at one time. Therefore, by adopting the method of batch submission, although the data can be successfully stored in the database, the storage time required is relatively long. By increasing the storage capacity and optimizing the storage structure, the methods of increasing the hard disk capacity and building distributed storage are adopted to increase the storage capacity. The effect is very obvious, but it only temporarily relieves the storage pressure. From the perspective of optimizing the storage structure, for the storage of homogeneous data, due to the reliance on relational databases, the partition and table method is adopted to effectively utilize the storage space. The method of optimizing from the storage structure upward lacks the overall management of the enterprise data that needs to be stored, and can only be stored and scheduled statically through the data results. As a result, by increasing the storage capacity and optimizing the storage structure, the cost, performance, and concurrency pressure brought by a large amount of data to data storage cannot be fundamentally solved. Therefore, it is very necessary to design a blockchain-based data storage optimization method to improve storage performance and reduce storage costs. Summary of the invention

[0003] The purpose of the present invention is to provide a data storage optimization method based on blockchain to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a data storage optimization method based on blockchain, applied to a data storage optimization system based on blockchain, the system comprising:

[0005] Storage policy mechanism module, used to establish and set storage policies and continuous mechanisms for optimizing storage;

[0006] A storage cost calculation module is used to calculate the storage cost caused by monitoring the real-time changes of storage quantity statistics and storage resource allocation service calculation according to the storage strategy;

[0007] Data storage optimization module, which is used to continuously optimize data storage throughout the data lifecycle by using object tags and lifecycle policies.

[0008] According to the above technical solution, the operating steps of the method include:

[0009] Step 1: Establish storage policies and ongoing mechanisms for optimizing storage;

[0010] Step 2: Calculate the storage cost caused by monitoring the real-time changes of storage statistics and storage resource allocation service calculation according to the storage policy;

[0011] Step 3: Use object tags and lifecycle policies to continuously optimize data storage throughout the data lifecycle.

[0012] Step 4: Synchronize the transfer optimization of the data storage layer based on the access frequency of the data storage block.

[0013] According to the above technical solution, the steps of establishing a blockchain-based enterprise data optimization storage strategy include:

[0014] Step 11: For each storage terminal stored in a single database of the cloud platform by the application system, statistics are performed on the storage data volume of different types of structured and semi-structured original storage data;

[0015] Step 12: By real-time monitoring of the available storage capacity of the storage end, the storage optimization threshold is set by using the ratio of the statistical storage quantity to the total storage capacity and the ratio of the available storage capacity to the total storage capacity;

[0016] Step 13: triggering a system storage optimization operation according to a set storage optimization threshold, and determining an allocation ratio of storage resources allocated to the storage optimization operation;

[0017] Step 14: Based on the available storage capacity and allocation ratio, the centralized raw data originally stored in a single database is sharded through the built distributed cluster, and distributed to each storage node in the distributed cluster through storage optimization operations. The mutual collaboration between each storage node is used to realize the processing and storage allocation of massive structured data, and the distribution is used to process and store unstructured data;

[0018] Step 15: Set up a continuous mechanism for storage: store the data in segments, then use the blockchain's storage chain to store the index of the complete data, and the sharing chain to store the index of the shared part of the data, so as to achieve blockchain capacity reduction. Users with different attributes request to share different parts from different blockchains according to their permissions.

[0019] According to the above technical solution, the step of calculating the real-time change of the storage quantity statistics by monitoring according to the storage strategy includes:

[0020] Enterprises store data sets in various storage terminals in a single database on the cloud platform. When storing data through cloud storage, the importance of the stored data in the workload and its operation is determined based on the access frequency of the stored data. Data audits are used to identify and catalog all stored data. The value of the data set is determined based on its importance to business operations, return on investment, and frequency of use. Access logs and usage analysis indicators are collected from data storage decisions. The storage cost of intermediate data generated after raw data analysis and calculation is calculated in the database, as well as the access cost formed by analysis and calculation and access;

[0021] The sum of the data storage costs in the cloud platform is the computing cost and storage cost of the data set specified by the cloud service provider. The factors affecting the storage cost are the response time in the cloud platform and the size of the stored data set. In cloud storage, select the intermediate data set with the greatest time impact stored in the cloud to reduce the shorter response time and the smaller total cost. As the number of stored data sets increases, the response time will decrease, that is, the computing cost will decrease, while the storage cost will increase. The total cost will fluctuate as the number of data sets stored increases.

[0022] According to the above technical solution, the step of calculating the storage cost caused by the storage resource allocation service calculation includes:

[0023] Setting k t is a binary variable, when k t = 1, indicating that the data object is in the storage layer of raw data storage, calculation and analysis at time t. t = 0, it means that the data object is in the intermediate data storage layer generated by the original data analysis and calculation at time t. The storage cost of a single data object storing a data set at time t is T1, T1 = k t *s1+(1-k t *s1)*s2,k t ∈{0.1}, where s1 is the storage cost price of the data in the storage layer of the original data storage calculation and analysis, s2 is the storage cost price of the data in the intermediate data storage layer generated by the original data analysis and calculation, and the access calculation generated by the intermediate data storage process formed by the original data calculation and analysis at time t for a single data object is T2, T2=k t *n1+(1-k t )*(n1+m2), k t∈{0.1}, where n1 is the number of accesses to the data when it is in the storage layer of the original data storage calculation and analysis, and m2 is the amount of retrieval of the stored data in the intermediate data storage layer generated by the original data analysis and calculation, that is, the access cost includes not only the access request operation cost, but also the retrieval cost. At time t, the access operation request cost of the data object is the product of the price of a single data access operation and the number of accesses, and the retrieval cost is the product of the unit price of data retrieval and the retrieval amount;

[0024] Recursively calculate the total storage cost of data stored in the time period set in the storage strategy, and calculate the storage cost of intermediate data stored in the database generated by the original data analysis and calculation. The total storage cost T generated includes: the storage cost T1 of the storage data set, and the cost T2 of the access calculation generated by the original data analysis and calculation to form the intermediate data storage process. The data storage cost calculation formula for a given storage strategy using the data storage cost model is: where tt n It is the storage cost calculation period specified in a given storage policy.

[0025] According to the above technical solution, the step of continuously optimizing data storage includes:

[0026] Categorize stored data sets based on data types, calculate the sensitivity and criticality of each data type based on the access patterns to stored data classified by different daily, weekly or monthly usage patterns, categorize and assign importance levels to data types based on sensitivity and importance;

[0027] Assess the importance of various types of data to workloads, determine the high, medium, and low priorities for stored data based on the data's value to the organization, regulatory requirements, and the potential impact of data loss, and apply tags at the stored file level to mark the sensitivity and criticality of each data set based on data structure and usage;

[0028] The original data is calculated and analyzed to generate data storage for intermediate data. There is an associated mapping between the original data and the intermediate data. The stored original data is combined into an importance model object, and the model object code is created. According to the data importance model, the storage cost is optimized for the properties of the intermediate data set generated by the original data calculation and analysis and the relationship between adjacent data sets. The formed storage space is classified into multiple levels, divided into individuals, departments, and enterprises according to the scope, and divided into structured and semi-structured data and unstructured data according to the category. The storage space is constructed, allocated, and scheduled for each category at each level to form a global general product baseline library to meet the data retrieval, mining, decision analysis applications of all product businesses of the enterprise, while providing data integration and services for the data storage life cycle.

[0029] According to the above technical solution, the step of optimizing the transfer of the data storage layer based on the access frequency of the data storage block includes:

[0030] According to the importance model of the data set, the importance value of each intermediate data is calculated, and the storage resource allocation of all stored data sets is sorted. The intermediate data sets with the highest importance value are placed in the storage layers with the highest storage level. All intermediate data sets are layered. When a certain intermediate data set is to be stored, the importance of the data sets associated with the intermediate data set will also change accordingly. Each time an intermediate data set is stored, its storage status is annotated in real time. The next intermediate data set to be stored is determined based on the annotated storage status. The initial status of all intermediate data sets is set to that none of the intermediate data sets are stored. The total cost, response time and storage cost of randomly storing intermediate data sets are given. According to the storage The access level of data stored in cloud storage during its life cycle is defined as hot access and cold access. When the access frequency of the stored intermediate data set is greater than or equal to the set access threshold, the access level of the data is defined as hot access, and the data continues to be stored in a storage layer with a higher storage level. When the access frequency of the stored intermediate data set is less than the set access threshold, the access level of the data is defined as cold access. When the access frequency of the data decreases with the growth of time, its storage layer is transferred from high to low. When the access frequency of the data increases with the growth of time, its storage layer is transferred from low to high. The storage layer transition decision is made based on the change of access frequency to save the overall cloud storage cost.

[0031] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention, by establishing and setting storage strategies and a continuous mechanism for optimizing storage, calculates the storage cost caused by monitoring the real-time changes of storage statistics and storage resource allocation service calculations according to the storage strategies, and then uses object tags and lifecycle strategies to continuously optimize data storage throughout the data lifecycle; and synchronously optimizes the transfer of data storage layers based on the access frequency of data storage blocks, so that in blockchain-based data cloud storage, intermediate data sets are used to make storage layer transition decisions for data through changes in access frequency. When the access frequency of data decreases with the increase of time, its storage layer is transferred from high to low, and when the access frequency of data increases with the increase of time, its storage layer is transferred from low to high. By making storage layer transition decisions for data through changes in access frequency, the overall cloud storage cost is saved and the storage performance of the system is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0033] Figure 1 A flowchart of a data storage optimization method based on blockchain provided in Example 1 of the present invention;

[0034] Figure 2 A schematic diagram of the module composition of a blockchain-based data storage optimization system provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Embodiment 1: Figure 1 This is a flowchart of a data storage optimization method based on blockchain provided in Example 1 of the present invention. This embodiment can be applied to the scenario of data storage performance optimization. The method can be executed by a data storage optimization system based on blockchain provided in this embodiment. Figure 1 As shown, the method specifically comprises the following steps:

[0037] Step 1: Establish storage policies and ongoing mechanisms for optimizing storage;

[0038] In an embodiment of the present invention, establishing a blockchain-based enterprise data optimization storage strategy includes:

[0039] Step 11: For each storage terminal stored in a single database of the cloud platform by the application system, statistics are performed on the storage data volume of different types of structured and semi-structured original storage data;

[0040] Step 12: By real-time monitoring of the available storage capacity of the storage end, the storage optimization threshold is set by using the ratio of the statistical storage quantity to the total storage capacity and the ratio of the available storage capacity to the total storage capacity;

[0041] Step 13: triggering a system storage optimization operation according to a set storage optimization threshold, and determining an allocation ratio of storage resources allocated to the storage optimization operation;

[0042] Step 14: Based on the available storage capacity and allocation ratio, the centralized raw data originally stored in a single database is sharded through the built distributed cluster, and distributed to each storage node in the distributed cluster through storage optimization operations. The mutual collaboration between each storage node is used to realize the processing and storage allocation of massive structured data, and the distribution is used to process and store unstructured data;

[0043] Step 15: Set up a continuous mechanism for storage: store the data in segments, then use the blockchain's storage chain to store the index of the complete data, and the sharing chain to store the index of the shared part of the data, so as to achieve blockchain capacity reduction. Users with different attributes request to share different parts from different blockchains according to their permissions.

[0044] Step 2: Calculate the storage cost caused by monitoring the real-time changes of storage statistics and storage resource allocation service calculation according to the storage policy;

[0045] In an embodiment of the present invention, an enterprise stores data sets in each storage terminal in a single database of a cloud platform. When storing data through cloud storage, the importance of the stored data in the workload and its operation is determined according to the access frequency of the stored data. Data audit is used to identify and catalog all stored data. The value of the data set is determined according to the importance of the data set to business operations, return on investment and frequency of use. Access logs and usage analysis indicators are collected from data storage decisions. The storage cost of storing intermediate data generated after analysis and calculation of the original data in the database and the access cost formed by analysis and calculation and access are calculated.

[0046] Exemplarily, the sum of the data storage costs in the cloud platform is the computing cost and storage cost of the data set specified by the cloud service provider. The factors affecting the storage cost are the response time in the cloud platform and the size of the stored data set. In cloud storage, the intermediate data set with the greatest time impact is selected and stored in the cloud, reducing the shorter response time and the smaller the total cost. As the number of stored data sets increases, the response time will decrease, that is, the computing cost will decrease, while the storage cost will increase, and the total cost will fluctuate as the number of data sets stored increases;

[0047] For example, we set k t is a binary variable, when k t = 1, indicating that the data object is in the storage layer of raw data storage, calculation and analysis at time t. t = 0, it means that the data object is in the intermediate data storage layer generated by the original data analysis and calculation at time t. The storage cost of a single data object storing a data set at time t is T1, T1 = k t *s1+(1-k t *s1)*s2,kt ∈{0.1}, where s1 is the storage cost price of the data in the storage layer of the original data storage calculation and analysis, s2 is the storage cost price of the data in the intermediate data storage layer generated by the original data analysis and calculation, and the access calculation generated by the intermediate data storage process formed by the original data calculation and analysis at time t for a single data object is T2, T2=k t *n1+(1-k t )*(n1+m2), k t ∈{0.1}, where n1 is the number of accesses to the data when it is in the storage layer of the original data storage calculation and analysis, and m2 is the amount of retrieval of the stored data in the intermediate data storage layer generated by the original data analysis and calculation, that is, the access cost includes not only the access request operation cost, but also the retrieval cost. At time t, the access operation request cost of the data object is the product of the price of a single data access operation and the number of accesses, and the retrieval cost is the product of the unit price of data retrieval and the retrieval amount;

[0048] Exemplarily, a recursive method is used to calculate the total storage cost of data stored in the time period set in the storage strategy, and the storage cost of intermediate data generated by the original data analysis and calculation in the database. The generated total storage cost T includes: the storage cost T1 of the storage data set, and the cost T2 of the access calculation generated by the original data analysis and calculation to form the intermediate data storage process. The data storage cost calculation formula for a given storage strategy using the data storage cost model is: where tt n It is the storage cost calculation period specified in a given storage policy.

[0049] Step 3: Use object tags and lifecycle policies to continuously optimize data storage throughout the data lifecycle.

[0050] In an embodiment of the present invention, the stored data sets are classified according to data types, the sensitivity and criticality of each data type is calculated based on the access patterns to the stored data according to different classifications of daily, weekly or monthly usage patterns, and the data types are classified and assigned importance levels according to the sensitivity and importance;

[0051] Assess the importance of various types of data to workloads, determine the high, medium, and low priorities for stored data based on the data's value to the organization, regulatory requirements, and the potential impact of data loss, and apply tags at the stored file level to mark the sensitivity and criticality of each data set based on data structure and usage;

[0052] The original data is calculated and analyzed to generate data storage for intermediate data. There is an associated mapping between the original data and the intermediate data. The stored original data is combined into an importance model object, and the model object code is created. According to the data importance model, the storage cost is optimized for the properties of the intermediate data set generated by the original data calculation and analysis and the relationship between adjacent data sets. The formed storage space is classified into multiple levels, divided into individuals, departments, and enterprises according to the scope, and divided into structured and semi-structured data and unstructured data according to the category. The storage space is constructed, allocated, and scheduled for each category at each level to form a global general product baseline library to meet the data retrieval, mining, decision analysis applications of all product businesses of the enterprise, while providing data integration and services for the data storage life cycle.

[0053] Step 4: Synchronize the transfer optimization of the data storage layer based on the access frequency of the data storage block;

[0054] In an embodiment of the present invention, according to the importance model of the data set, the importance value of each intermediate data is calculated, the storage resource allocation of all stored data sets is sorted, and the intermediate data sets with the importance values ​​from high to low are placed in the storage layers with the storage levels from high to low, and all the intermediate data sets are layered. When a certain intermediate data set is to be stored, the importance of the data sets associated with the intermediate data set will also change accordingly. Each time an intermediate data set is stored, its storage status is marked in real time, and the next intermediate data set to be stored is determined according to the marked storage status. The initial status of all intermediate data sets is set to that all intermediate data sets are not stored, and the total cost, response time and storage cost of randomly storing intermediate data sets are given. , define hot access and cold access according to the access degree of data stored in cloud storage during its life cycle. When the access frequency of the stored intermediate data set is greater than or equal to the set access threshold, the access degree of the data is defined as hot access, and the data continues to be stored in a storage layer with a higher storage level. When the access frequency of the stored intermediate data set is less than the set access threshold, the access degree of the data is defined as cold access. When the access frequency of the data decreases with the growth of time, its storage layer is transferred from high to low. When the access frequency of the data increases with the growth of time, its storage layer is transferred from low to high. The storage layer transition decision is made based on the change of access frequency, saving the overall cloud storage cost.

[0055] Embodiment 2: Embodiment 2 of the present invention provides a data storage optimization system based on blockchain. Figure 2 A schematic diagram of the module composition of a data storage optimization system based on blockchain provided in Embodiment 2 of the present invention, such as Figure 2 As shown, the system includes:

[0056] Storage policy mechanism module, used to establish and set storage policies and continuous mechanisms for optimizing storage;

[0057] A storage cost calculation module is used to calculate the storage cost caused by monitoring the real-time changes of storage quantity statistics and storage resource allocation service calculation according to the storage strategy;

[0058] Data storage optimization module, which is used to continuously optimize data storage throughout the data lifecycle by using object tags and lifecycle policies.

[0059] In some embodiments of the present invention, the storage policy mechanism module includes:

[0060] A storage data volume statistics module is used to perform storage data volume statistics on different types of structured and semi-structured raw storage data;

[0061] A storage optimization threshold setting module is used to set a storage optimization threshold by using a ratio of a statistical storage quantity to a total storage quantity and a ratio of an available storage quantity to a total storage quantity;

[0062] The storage persistence mechanism setting module is used to set the persistence mechanism for storage.

[0063] In some embodiments of the present invention, the storage cost calculation module includes:

[0064] An access frequency acquisition module, used to acquire the access frequency of the stored data to determine its importance in the workload and its operation;

[0065] An intermediate data storage cost calculation module is used to calculate the storage cost of storing the intermediate data generated after the original data analysis and calculation in the database;

[0066] An access cost calculation module is used to calculate the access cost generated by the analysis and calculation of the original data and the analysis and calculation and access of the intermediate data stored in the database;

[0067] The total storage cost calculation module is used to calculate the data storage cost under a given storage strategy using a data storage cost model.

[0068] In some embodiments of the present invention, the data storage optimization module includes:

[0069] An importance level assignment module for classifying and assigning importance levels to data types based on sensitivity and importance;

[0070] Importance model object module, used to combine the stored raw data into importance model objects;

[0071] an access degree definition module, used to define cold access and cold access according to the access degree of data stored on the cloud storage during its life cycle;

[0072] The storage transfer optimization module is used to optimize the transfer of the data storage layer based on the access frequency of the data storage block.

[0073] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0074] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data storage optimization method based on blockchain, applied to a data storage optimization system based on blockchain, characterized in that: The system comprises: Storage policy mechanism module, used to establish and set storage policies and continuous mechanisms for optimizing storage; A storage cost calculation module is used to calculate the storage cost caused by monitoring the real-time changes of storage quantity statistics and storage resource allocation service calculation according to the storage strategy; Data storage optimization module, which is used to continuously optimize data storage throughout the data lifecycle by using object tags and lifecycle policies.

2. According to a blockchain-based data storage optimization method according to claim 1, it is characterized in that: The storage policy mechanism module includes: A storage data volume statistics module is used to perform storage data volume statistics on different types of structured and semi-structured raw storage data; A storage optimization threshold setting module is used to set a storage optimization threshold by using a ratio of a statistical storage quantity to a total storage quantity and a ratio of an available storage quantity to a total storage quantity; The storage persistence mechanism setting module is used to set the persistence mechanism for storage.

3. A data storage optimization method based on blockchain according to claim 2, characterized in that: The storage cost calculation module includes: An access frequency acquisition module, used to acquire the access frequency of the stored data to determine its importance in the workload and its operation; An intermediate data storage cost calculation module is used to calculate the storage cost of storing the intermediate data generated after the original data analysis and calculation in the database; An access cost calculation module is used to calculate the access cost generated by the analysis and calculation of the original data and the analysis and calculation and access of the intermediate data stored in the database; The total storage cost calculation module is used to calculate the data storage cost under a given storage strategy using a data storage cost model.

4. A data storage optimization method based on blockchain according to claim 3, characterized in that: The data storage optimization module includes: An importance level assignment module for classifying and assigning importance levels to data types based on sensitivity and importance; Importance model object module, used to combine the stored raw data into importance model objects; an access degree definition module, used to define cold access and cold access according to the access degree of data stored on the cloud storage during its life cycle; The storage transfer optimization module is used to optimize the transfer of the data storage layer based on the access frequency of the data storage block.

5. A data storage optimization method based on blockchain according to claim 4, characterized in that: The operating steps of the method include: Step 1: Establish storage policies and ongoing mechanisms for optimizing storage; Step 2: Calculate the storage cost caused by monitoring the real-time changes of storage statistics and storage resource allocation service calculation according to the storage policy; Step 3: Use object tags and lifecycle policies to continuously optimize data storage throughout the data lifecycle. Step 4: Synchronize the transfer optimization of the data storage layer based on the access frequency of the data storage block.

6. A data storage optimization method based on blockchain according to claim 5, characterized in that: The steps of establishing a blockchain-based enterprise data optimization storage strategy include: Step 11: For each storage terminal stored in a single database of the cloud platform by the application system, statistics are performed on the storage data volume of different types of structured and semi-structured original storage data; Step 12: By real-time monitoring of the available storage capacity of the storage end, the storage optimization threshold is set by using the ratio of the statistical storage quantity to the total storage capacity and the ratio of the available storage capacity to the total storage capacity; Step 13: triggering a system storage optimization operation according to a set storage optimization threshold, and determining an allocation ratio of storage resources allocated to the storage optimization operation; Step 14: Based on the available storage capacity and allocation ratio, the centralized raw data originally stored in a single database is sharded through the built distributed cluster, and distributed to each storage node in the distributed cluster through storage optimization operations. The mutual collaboration between each storage node is used to realize the processing and storage allocation of massive structured data, and the distribution is used to process and store unstructured data; Step 15: Set up a continuous mechanism for storage: store the data in segments, then use the blockchain's storage chain to store the index of the complete data, and the sharing chain to store the index of the shared part of the data, so as to achieve blockchain capacity reduction. Users with different attributes request to share different parts from different blockchains according to their permissions.

7. A data storage optimization method based on blockchain according to claim 6, characterized in that: The step of calculating the real-time change of the storage quantity statistics by monitoring according to the storage strategy includes: Enterprises store data sets in various storage terminals in a single database on the cloud platform. When storing data through cloud storage, the importance of the stored data in the workload and its operation is determined based on the access frequency of the stored data. Data audits are used to identify and catalog all stored data. The value of the data set is determined based on its importance to business operations, return on investment, and frequency of use. Access logs and usage analysis indicators are collected from data storage decisions. The storage cost of intermediate data generated after raw data analysis and calculation is calculated in the database, as well as the access cost formed by analysis and calculation and access; The sum of the data storage costs in the cloud platform is the computing cost and storage cost of the data set specified by the cloud service provider. The factors affecting the storage cost are the response time in the cloud platform and the size of the stored data set. In cloud storage, select the intermediate data set with the greatest time impact stored in the cloud to reduce the shorter response time and the smaller total cost. As the number of stored data sets increases, the response time will decrease, that is, the computing cost will decrease, while the storage cost will increase. The total cost will fluctuate as the number of data sets stored increases.

8. A data storage optimization method based on blockchain according to claim 7, characterized in that: The step of calculating the storage cost caused by the storage resource allocation service calculation includes: Setting k t is a binary variable, when k t = 1, indicating that the data object is in the storage layer of raw data storage, calculation and analysis at time t. t = 0, it means that the data object is in the intermediate data storage layer generated by the original data analysis and calculation at time t. The storage cost of a single data object storing a data set at time t is T1, T1 = k t *s1+(1-k t *s1)*s2,k t ∈{0.1}, where s1 is the storage cost price of the data in the storage layer of the original data storage calculation and analysis, s2 is the storage cost price of the data in the intermediate data storage layer generated by the original data analysis and calculation, and the access calculation generated by the intermediate data storage process formed by the original data calculation and analysis at time t for a single data object is T2, T2=k t *n1+(1-k t )*(n1+m2), k t ∈{0.1}, where n1 is the number of accesses to the data when it is in the storage layer of the original data storage calculation and analysis, and m2 is the amount of retrieval of the stored data in the intermediate data storage layer generated by the original data analysis and calculation, that is, the access cost includes not only the access request operation cost, but also the retrieval cost. At time t, the access operation request cost of the data object is the product of the price of a single data access operation and the number of accesses, and the retrieval cost is the product of the unit price of data retrieval and the retrieval amount; Recursively calculate the total storage cost of data stored in the time period set in the storage strategy, and calculate the storage cost of intermediate data stored in the database generated by the original data analysis and calculation. The total storage cost T generated includes: the storage cost T1 of the storage data set, and the cost T2 of the access calculation generated by the original data analysis and calculation to form the intermediate data storage process. The data storage cost calculation formula for a given storage strategy using the data storage cost model is: where tt n It is the storage cost calculation period specified in a given storage policy.

9. A data storage optimization method based on blockchain according to claim 8, characterized in that: The step of continuously optimizing data storage comprises: Categorize stored data sets based on data types, calculate the sensitivity and criticality of each data type based on the access patterns to stored data classified by different daily, weekly or monthly usage patterns, categorize and assign importance levels to data types based on sensitivity and importance; Assess the importance of various types of data to workloads, determine the high, medium, and low priorities for stored data based on the data's value to the organization, regulatory requirements, and the potential impact of data loss, and apply tags at the stored file level to mark the sensitivity and criticality of each data set based on data structure and usage; The original data is calculated and analyzed to generate data storage for intermediate data. There is an associated mapping between the original data and the intermediate data. The stored original data is combined into an importance model object, and the model object code is created. According to the data importance model, the storage cost is optimized for the properties of the intermediate data set generated by the original data calculation and analysis and the relationship between adjacent data sets. The formed storage space is classified into multiple levels, divided into individuals, departments, and enterprises according to the scope, and divided into structured and semi-structured data and unstructured data according to the category. The storage space is constructed, allocated, and scheduled for each category at each level to form a global general product baseline library to meet the data retrieval, mining, decision analysis applications of all product businesses of the enterprise, while providing data integration and services for the data storage life cycle.

10. A data storage optimization method based on blockchain according to claim 9, characterized in that: The step of optimizing the transfer of the data storage layer based on the access frequency of the data storage block includes: According to the importance model of the data set, the importance value of each intermediate data is calculated, and the storage resource allocation of all stored data sets is sorted. The intermediate data sets with the highest importance value are placed in the storage layers with the highest storage level. All intermediate data sets are layered. When a certain intermediate data set is to be stored, the importance of the data sets associated with the intermediate data set will also change accordingly. Each time an intermediate data set is stored, its storage status is annotated in real time. The next intermediate data set to be stored is determined based on the annotated storage status. The initial status of all intermediate data sets is set to that none of the intermediate data sets are stored. The total cost, response time and storage cost of randomly storing intermediate data sets are given. According to the storage The access level of data stored in cloud storage during its life cycle is defined as hot access and cold access. When the access frequency of the stored intermediate data set is greater than or equal to the set access threshold, the access level of the data is defined as hot access, and the data continues to be stored in a storage layer with a higher storage level. When the access frequency of the stored intermediate data set is less than the set access threshold, the access level of the data is defined as cold access. When the access frequency of the data decreases with the growth of time, its storage layer is transferred from high to low. When the access frequency of the data increases with the growth of time, its storage layer is transferred from low to high. The storage layer transition decision is made based on the change of access frequency to save the overall cloud storage cost.

Images